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HOMOGENEOUS FUNCTIONS ON LIGHT CONES: 
THE INFINITESIMAL STRUCTURE OF SOME DEGENERATE 
PRINCIPAL SERIES REPRESENTATIONS 

roger e. howe and eng-chye tan 
1. Introduction 

One of the classic and most pleasing calculations in the representation theory of 
semisimple Lie groups is Bargmann's [Ba] (see also [Na]) description of the structure 
of representations of SL(2, R). Bargmann constructs a basis of eigenvectors for the 
compact subgroup S0(2) C SL(2,R) and then explicitly describes the action of 
the Lie algebra of SL(2,R) on this basis. From the straightforward and explicit 
formulas that result, it is easy to make conclusions about unitarity, irreducibility, 
isomorphism, and other natural questions concerning these representations. 

Unfortunately Bargmann's example stands almost alone; the literature tends 
to give the impression that once beyond the civilized confines of SL(2,R), one 
immediately finds oneself in a dense thicket of difficulties, where explicit calculation 
is difficult and of little value. The purpose of this paper is to point to an infinite 
class of examples in which the pleasant characteristics of Bargmann's work can be 
retained; one can describe a model for representations in which the action of the 
Lie algebra is given by explicit simple formulas, which permit direct determination 
of important structural properties of the representations. 

The examples we will treat are spaces of functions that live on light cones and 
satisfy appropriate homogeneity conditions. These spaces are modules for an ap- 
propriate isometry group — 0(p, q), U(p, q), or Sp(p, q). In case q = l, the spherical 
principal series are among our examples, and Kostant's celebrated results [Ko] on 
the complementary series can be read off from our formulas, excluding the one ex- 
ceptional rank one group. (Essentially this calculation was done, in less systematic 
fashion, in [Jo, JW].) 

We are hardly the first to study these representations. Significant parts of our 
results, including Dixmier's [Di] study of the De Sitter group, Hirai's notes on 
0(p, 1) and U(p, 1) [Hil, Hi2], Kostant's work [Ko], the calculations of Johnson 
and Wallach [JW] for the spherical principal series, the studies of Schlichtkrull [Sc] , 
Sekiguchi [Se], Strichartz [Stl, St2], Rallis and Schiffmann [RS], Rossmann [Ro], 
Molcanov [Mo], Faraut [Fa] 

on functions on hyperboloids, the calculations of Klimyk and various colleagues 
(see [KG] and the references therein) and others [Br, CI, Cw, FR, GN, HC1, JV, 
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Ni, Sa, Spl, Tk, Vi] are already in the literature. The paper [Br] has diagrams 
that resemble some of ours. The unitary duals of the rank one groups have been 
classified (see [BS, CI, Hil, Hi2] and the references therein). In fact, most or all of 
our results may be in the literature, at least implicitly. The main purpose of the 
present paper is to make our results explicit and to emphasize how concrete and 
how simple a picture one may draw of these representations. We use virtually no 
technology. There is no clear reason why these calculations could not have been 
done in the 1940s or early 1950s. If they had been, the development of semisimplc 
representation theory might have been substantially different. For instance, besides 
the results of Kostant mentioned above, it is easy to read off from our results the 
unitary duals of 0(2, 1), 0(4, 1), and U(2, 1). The last of these was not known until 
the mid 1960s [Hi2]. 

Here is a summary of the paper. In §2 we set the context and describe our 
representations in the case of the indefinite orthogonal groups 0(p, q). We describe 
the if-spectrum of the representations, where K = 0(p) x 0(q) is the maximal 
compact subgroup of 0(p, q), and we compute the infinitesimal action of the Lie 
algebra o(p,q) in terms of the X-spectrum. On the basis of the formulas for the 
o(p,q) action, we determine composition series for our representations. It turns 
out that the structure of the composition series involves some integrality conditions 
which make it depend in a significant way on the parities of p and q. Also, we must 
distinguish the cases when p or q is less than or equal to 2, since these involve some 
degeneration in the if-spectrum that affects the representation structure. 

In §3 we apply our formulas to determine which of our representations carry 
invariant Hcrmitian forms, and on which subquotients these forms can be definite, 
that is, which constituents of our representations are unitary. In particular, we 
determine the extent of the complementary series and what happens at the end of 
the complementary series. 

In §§4 and 5 we adapt the discussion of §§2, 3 to the unitary groups U(p, q) and 
the indefinite symplcctic groups Sp(p, q). The overall features of the picture for 
these groups is controlled by 0(p, q), as is made clear by Lemmas 4.1 and 5.3, but 
differences in the if -spectrum in the degenerate cases when q — 1 lead to interesting 
new phenomena, notably considerable variation in the length of the complementary 
series, including Kostant's results concerning the spherical complementary series of 
U(p,l) and Sp(p, 1). 

In order not to distract from the coherence and directness of the phenomena 
that we present, we omit the details of various calculations in §§2-5. These are 
worked out in §6. The final §7 discusses how our results fit into the general theory 
of representations of semisimple Lie groups. 

To conclude this introduction, we note that those readers who are familiar with 
the theory of reductive dual pairs [Ho3] will find our results can be profitably 
interpreted in terms of this theory, which can be used to predict much of what 
we present. We emphasize again, however, that our elementary and computational 
approach is independent of this or any general theory (except that we use [Ho3] as 
a convenient reference for facts about spherical harmonics that must somewhere be 
in the classical literature). 
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2. THE 0(p, g)-MODULE STRUCTURE of HOMOGENEOUS FUNCTIONS 
ON LIGHT CONES: if- TYPES AND COMPOSITION SERIES 



Let RP+i 



denote the vector space of tuples w — (x, y) where 



and y 



To fix ideas, we will take p > q. We consider W +g to be endowed with the indefinite 
inner product 



(w,w')p. q = {x,x') p - {y,y') q 



3 = 1 



3 = 1 



We will use the notation 



r LK> = (w,w) 



and likewise for ^(x), r^(y). 

Let 0(p, q) denote the isometry group of (•, -)p,9i that is, 0(p, q) is the subgroup 
of linear transformations g £ GL(p + q, M) such that 

(gw, gw') Pt q = (w, w') p . q , w,w' G R. p+q . 

Clearly the group 0(p), the isometry group of (•,•)? on M p , can be regarded as a 
subgroup of 0(p,q) by letting it act on the x component of w — (x,y). Similarly, 
0(g), the isometry group of (•, on R ? , can be considered a subgroup of 0(p, q). 
Combining these, we get an embedding of 0(p) xO(q) C 0(p, q), as the subgroup of 
0(p, q), which stabilizes the decomposition W +q ~ R P ©R 9 implicit in the notation 
w = (x,y). The group 0(p) x 0(q) is clearly a compact subgroup of 0(p, q). It is 
well known and not hard to see that 0(p) x 0(q) is a maximal compact subgroup 
ofO(p,«). 

Let M m , n (F) denote the space of to x n matrices over a field F; if m = n, 
we shall abbreviate M m ^ m (F) by M m (F). We denote by o(p,q) the Lie algebra of 
0(p, q), the space of (p + q) x (p + q) matrices T G M p+q (R), which are generators of 



one-parameter subgroups {e 



sT 



s e 



of 0(p, q). By differentiating the relation 



(e s w,e s w') p ^ q = (w,w') Pi 



9' 

we obtain the description 
(2.1) 

o(p,q) = {T G M P+9 (R)| (r«),iB') M + («;,r«/)p,, = 

for all w, w G 



A = -A* G Mp(R); C = -C* G M 9 (R); B G M. 



r ™(R)}- 



Here A* denotes the transpose of the matrix A. 
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Consider the light cone 

K., = X° = {we - {0} r* » = 0| 
(2-2) , L . J 

= {{x,y)eW + «-{0} rl(x)=r 2 q (y)\. 

Observe that we exclude the origin from X°. 

The action of 0(p, q) on M. p+q preserves the light cone X°. It is well known (in 
particular, it is a direct consequence of Witt's Theorem [Ja]) that X° consists of 
a single orbit for 0(p, q), that is, the action of 0(p, q) on X° is transitive. Thus if 
w\ is a specified point in X° and Qi C 0(p,q) is the subgroup that stabilizes w\, 
then the map g — > gwi defines a bijection (in fact, a diffeomorphism) 

a: 0(p,q)/Q 1 ~X°. 

Let Pi C 0(p,q) be the stabilizer of the line M.w\ through w\. Then Pi is a 
parabolic subgroup [Kn] of 0(p, q), and a (Pi) = R x ?«i; more precisely, the map 
p — > a(p) where pw\ = a(p)u>\ defines a group isomorphism 

(2.3) a : P 1 /Q 1 ~R X . 

The second description of X° in (2.2) allows us to analyze the action of 0(p) x 0(q) 
on X°. Let 

gf- 1 = {x g M p I (a;,a;) p = 1} 

be the unit sphere in R p . It is a homogeneous space for 0(p). We have on X° an 
analog of "polar coordinates" . 

Scholium 2.1. The mapping 

(2.4) (3 : X° -> S 13 - 1 x S^ 1 x M x 
defined by 

W = {X ' V) (r2(x)V2' ^V2' r p( X ) 1/2 ) 

is a diffeomorphism. The level sets of r 2 p {x) are 0(p) x O(q) orbits, equivalent via 
a scalar dilation to x 

The action of 0(p,q) on M. p+q can be used to define an action of 0(p,q) on 
functions on W }+q by the familiar formula 

(2.5) P(g)(f)(w) - fig-'w), g e 0(p, «), «; G 

for a function / on R p+<? . Precisely the same formula defines an action on functions 
on any set X C W +q that is invariant under 0(p, q). Furthermore, the restriction 
mapping rx '■ f — > fi taking a function / to its restriction to X, clearly commutes 
with the 0(p,q) action. This means that if we have a function / on X, we can 
compute p(g)(f) by extending / to some function / on all of R p+9 , computing 
p{g)(f), then restricting back to X. This compatibility justifies at least partially 
our abuse of notation in failing to specify on what set / is defined in formula (2.5). 
We will be interested in taking X = X°, the light cone. 
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If we differentiate the action (2.5) along one-parameter subgroups of 0(p, q), we 
obtain a representation of the Lie algebra o(j>, q) on functions via differential oper- 
ators. Straightforward computation shows that the space of operators representing 
o(p,q) is spanned by the first-order operators: 

1 < j < k < p, 

(2.6) k fm = yi-^:. — y™"£r' 1 < z < m < <?, 

1<3<P, l<l<q- 

Here the £;j fc describe the action of the Lie subalgebra o(p) C o(p, q) corresponding 
to the subgroup 0(p) of 0(j>, q). Similarly the kf m describe the action of o(q). The 
Pjl span a complement to o(p) © o(q) in o(p,q), corresponding to the off-diagonal 
matrix B in formula (2.1). We denote the span of the Pji by p. 

We propose to study spaces of homogeneous functions on the light cone X°. 
Thus for a e C denote by S a (X°) the space 

(2.7) s a (x°) = {/ e c°°ps: )| f{tw) = t a f{w), wex°, teR^} 

of smooth functions on X° homogeneous of degree a under dilations by positive 
scalars. Since 0(p, q) commutes with scalar dilations, it is clear that S a (X°) will 
be invariant under the action (2.5) of 0(p,q). Indeed there is no need to consider 
only positive dilations. We could further define two subspaces 

s a± (x°) = {/ e s a (x°)\ f(-w) = ±/( w )} . 

It is clear that 

(2.8) S a {X ) = S a+ (X°)®S a -(X°) 

and that each of S a+ (X°) and S a ~(X°) are individually invariant under 0(p, q). 
Thus it would be possible, and might seem advisable, to study the S a+ (X°) and 
the S a ~(X°) separately. However, we will find it convenient and enlightening to 
lump S a± (X°) together in the space S a (X°). 

Before proceeding to explicit computations, we observe that the spaces S a± (X°) 
are naturally identifiable as 0(p, q) modules to certain induced representations, of a 
type known informally as "degenerate principal series" . Recall the homomorphism 
a : Pi — > M x of formula (2.3). For a e C, and a choice ± of sign, define a 
quasicharacter (= homomorphism to C x ) of P\ by 



(2-9) V±(P) 



a(p) a if a(p) > 0, 

±\&(p)\ a if d(p)<0. 



Define a map 

(2.10) S : S a± (X°) -» C°°(G) 



by the formula 



S(f)(g) = P(g)(f)(wi) = fig-'wi). 
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Hero wi G X° is the point used to define Pi. 
Straightforward formal checking shows that 

6{f)(pg) = , v e Pi, g e G, 

[ ■ ' 5(f)(gh)=5(p(h)f)(g). 

These formulas, plus the definition of Pi, imply that S defines an equivalence of 
0(p,q) representations between S a± (X°) and the representation ind Pl (V^) -1 in- 
duced from the character (V^) -1 of Pi [Kn]. (Here we are using unnormalized 
induction.) If q = 1 and we choose + from ±, these representations are the spheri- 
cal principal series of 0(p, 1) [Kn]. 

Now let us investigate the structure of the S a (X°) as 0(p,q) modules. First 
consider the action of the compact subgroup 0(p) x 0(g). The decomposition 
(2.4) of X° shows that any function in S a (X°) is determined by its restriction to 
0-1(^-1 x §9-1 x {i}). More exactly, restriction to /T^S 35 " 1 x S 9 " 1 x {1}) of 
elements of S a (X°) defines an isomorphism of 0(p) x 0(g) modules. 

The classical theory of spherical harmonics [Ho3, Vi, Zh] describes C°°(E: P ~ 1 ) as 
a representation of 0(p). It is the (topological) direct sum of the finite-dimensional 
spaces Tt m (M. p ) consisting of the (restrictions to § p_1 of the) harmonic polynomials 
of degree m, m € Z + ; 

(2.12) C 00 ^- 1 ) ~ ^ m (K P )- 

m>0 

The space H m (R p ) consists of polynomials of degree m which are annihilated by 
the Laplace operator 

3 = 1 3 

Let V m {W) denote the space of polynomials of degree m on W. Then the Laplace 
operator maps V m {R p ) to V m - 2 (W): that is, 

(2.13) A p : V m {W) -» T m - 2 {W), 

with kernel H m (W). It is part of the theory of spherical harmonics that the map- 
ping (2.13) is surjective; hence 

dimH m (W) = dimV m (R p ) - dirnV" 1 - 2 



m+p— l\ fm+p — 3 
p-l )-{ p-l 

If p > 1, this is always nonzero, but if p = 1, it is zero for m > 2. Thus although 
the sum (2.12) always is nominally over all of Z+, for p — 1, it is effectively only 
over m = 0, 1. Each H m (M p ) defines an irreducible representation of 0(p), and the 
representations defined by two different 7i m (W) are inequivalcnt. 

Combining the decompositions (2.12) for p and for q, we obtain a decomposition 

(2.14) C 00 ^" 1 x S"' 1 ) ~ n m (W)»H n (R q ) 

m.n>0 
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of the functions on S p_1 x into irreducible incquivalcnt subspaces for 0(p) x 
0(g). The above remarks about restriction to /3~ 1 (S P_1 x § 9_1 x {1}) show that 
equation (2.14) also yields a description of the 0(p) x 0(g)-module structure of 
S a (X°). The main point to observe is that we can adjust the homogeneity of 
functions by multiplication by powers of r 2 (or r 2 ) without changing the way they 
transform under 0(p) x O(q). Thus we can define an embedding 

(2.15) Ja , m .n = Ja : H m (W) ® H n (W) -> S a (X°) 
by the formula 

j a (hi ®h 2 ){x,y) = hi(x)h 2 (y)r 2 p (x) h , 

where h x e H m {W), h 2 G H"(K«), and b is chosen so that the total degree of 
homogeneity is a, that is, m + n + 2b = a. In formula (2.15) we are implicitly 
restricting hih 2 r 2 , b to X° . The following statement summarizes our discussion of 
the 0(p) x 0(g) action. 

Lemma 2.2. As an 0(p) x 0(g) module, the space S a (X°) decomposes into a direct 
sum 

(2.16) S a {X°)~ Ja(H m (R p )®H n {R q )) 

m,n>0 

of mutually inequivalent irreducible representations. 

We will refer to the spaces j a (H m (W) <g) H n (R q )) as the .RT-types of S a (X°). 
We remind the reader that in case p (or g) equals 1, the sum over m (or n) is limited 
to m (or n) equal to or 1. 

Lemma 2.2 tells us the structure of S a (X°) as an 0(p) x 0(g) module. Since the 
if-types all define distinct 0(p)xO(g) subrepresentations, any 0(p) xO(g)-invariant 
subspacc of S a (X°) will be a sum of the -ftT-types it contains. This will in particular 
be true of any 0(p, g)-invariant subspacc of S a (X°). Thus, to understand the 
0(p, g)-module structure of S a (X°), we need to see how 0(p, g) transforms one K- 
type to another. For this purpose, it is helpful to represent the if-types graphically, 
as points in the plane M 2 . Precisely, we will represent j a {H m {W) ® H n (R q )) as the 
point with integer coordinate (m, n) in the positive quadrant (M+) 2 in R 2 . Thus the 
set of all K- types is matched to the set of all integer points in the positive quadrant, 
as illustrated by Diagram 2.17 (see next page). (This picture is for p, q > 2, which 
we will assume for the present. We will treat the degenerate case g — 1 later.) We 
have used two different symbols to mark the points to remind the reader of the 
decomposition of S a (X a ) into S a± (X°). The dots indicate K-types in S a+ (X a ) 
(the parity of m + n is even), and the crosses indicate K- types in S a ~(X°) (the 
parity of m + n is odd). Sometimes we will indicate a K-type j a (H m (W)(g>H n (R q )) 
simply by its associated coordinates (m, n). 
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X • X • X 

• X • X • 
X • X • X 

• X • X • 



-X- 



-X- 



-X- 



Legend: • if -types of S a+ (X°) 
x if -types of S a - (X") 



Diagram 2.17 

To go beyond the if -type structure of S a (X°), we will investigate the effect of 
applying elements from the subspace p of the Lie algebra o(p,q) to the if -types. 
Since the if- types are individually invariant under o(p) © o(q), any collection of 
if -types such that any one of them is mapped by p into the span of the others will 
define (i.e., its span will be) an o(p, q) submodule of S a (X°). An early theorem of 
Harish-Chandra [HC2] guarantees that the closure of this span in S a (X°) will be an 
0(p, q) submodule. Thus our computation of the action of p on individual if -types 
is in principle (and will turn out to be in practice) sufficient for understanding the 
submodule structure of S a (X°). 

The operators of p are described in the formula (2.6). A computation, which 
will be carried out in §6.1, establishes the following rules. 



Lemma 2.3. For each pair (m,n), there are maps 

T tt ■ p ® (n m (R p ) ® n n (R q )) -> n m±1 ( 



which are independent of a and which are nonzero as long as the target space is 
nonzero, such that the action of z e p on the K-type j a (H m (W) <g> H n {R q )) is 
described by the formula 



(2.18) 



p(z)j a (<f>) =(a-m- n) Ja (T++(z ® <P)) 

+ (a-m + n + q- 2)j a {T+~(z ® <f>)) 
+ (a + m-n + p- 2)j a (T m + (z ® <j>)) 
+ (a + m + n+p + q- A)j a {T~{z ® 4>)), 
where (j)eH m {W)® H n (R q ). 



Proof. See §6.1. □ 



In terms of the graphical representation (see Diagram 2.17) of the if -types, we 
may picture this formula as follows. Starting from a given point (m, n), the action 
of p potentially can take us to any of the diagonally adjacent points (m ±1, nil). 
Whether we can, in a given S a (X°), actually proceed from (m, n) to a given one of 
the points (ra ± 1, n ±1) depends on whether the coefficient of the operator 
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(e.g., a — to — n for T++) in the formula (2.18) is nonzero or not. We can represent 
the situation graphically as in Diagram 2.19. 

(m-l,n+l) (m + l,n+l) 




(m — l,n— 1) (m+l,n— 1) 

Diagram 2.19 

The four corner points in Diagram 2.19 are the K-types we may hope to reach 
from (m, n) by the action of p; whether we actually can get to a given one of the 
points depends on whether the transition coefficient A^ a {m,n) given by 

A pXa,( m > n ) =a-m-n, 
At^fm, n)=a-m + n + q-2, 

(2.20) 

A P Xa( m > n ) = a + m-n+p-2, 

A p ^a( m > n)=a + m + n+ p + q-4, 

is nonzero. We shall omit the subscripts in A^ if it is clear from context. 

Remark. We see that from a given (m, n) by arbitrarily repeated action of p we 
can never hope to get to all other points (to/, n') — only to those such that to' + n' 
has the same parity as m + n. This reflects the fact, already observed, that S a (X°) 
breaks up into the two invariant subspaces S a± (X°). 

We see that if a is not an integer, then the transition coefficients A ±:t are never 
zero for any (m,n) in (Z + ) 2 . From this and Diagram 2.19, we see that, starting 
from any if- type j a (H m (W) ® H n (M. q )), we may by sufficiently many applications 
of elements from p eventually reach any other if- type corresponding to (m',n'), 
providing ml + n' has the same parity as (m, n). Combining this with the de- 
scriptions (2.17), we can make our first conclusion concerning reducibility of the 
S a (X a ). 

Proposition 2.4. If a is not an integer, then each of S a± (X a ) is an irreducible 
0(p,q) module. 

Consider now what happens when a is an integer. Suppose a > 0, and consider 
a K-type (m, n) such that m + n = a. This means A ++ = 0, so we cannot 
move from (m, n) to (m + l,n + 1) by acting by p. We can perhaps move to 
(m', n') = (m + 1, n — 1) or to (to', n') = (m — 1, n + 1), but from these points we 
again cannot move to the upper right, as seen in terms of Diagram 2.19 since again 
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m' + n' = a. We can see that the action of p, no matter how many times repeated, 
will never take us above the line to + n = a. Thus the set of if-types (m, n) with 
to + n < a and m + n = a (mod 2) form an o(p, q) submodule of S a (X°). 

We interpret this using our graphical representation of if -types as follows. We 
think of the line a = x + y in R 2 as being a barrier, which moves as a varies. When 
this moving line passes through integer points in the positive quadrant, it blocks 
attempts to move past it from those if- types (of the correct parity); thus, trapping 
all if-types below it in an o(p, q) submodule. This is illustrated by Diagram 2.21. 



y 




X 



a 

Diagram 2.21 



Similar reasoning applies to each of the transition coefficients A ±zt . Each of 
them can be thought of as defining a line in R 2 , namely, the set of (x, y) € R 2 that 
satisfy A ±:iz (x,y) = 0. We will abuse notation and use the symbol A ±=t to refer 
to the line or barrier defined by the zero locus of the linear functional A ±± (x 1 y) 
defined in (2.20). These lines move as a varies, and when they pass through integer 
points (which they all do simultaneously, when a is an integer), they trap the K- 
types on one side and so give rise to a submodule. The position of these barriers 
for a typical a > is given in the Diagram 2.22. 
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A~+ : y = x + a + p-2 




Diagram 2.22 

The arrows on each barrier point in the direction of the submodule defined by 
the barrier. Observe that the barriers A ++ and A arc parallel and the associated 
arrows point in opposite directions. Likewise the barriers A H and A h arc parallel 
to each other and perpendicular to the other two. As a varies, the barriers A ++ 
and A A move so that they intersect the x-axis at points q — 2 units apart, and 
the barriers A ++ and A~ + always intersect the y-axis at points p — 2 units apart. 
Thus A ++ never intersects A + ~ or A h inside the positive quadrant unless p or q 
equals 2. Also, the barriers A ++ and A~~ never intersect the positive quadrant at 
the same time. They coincide when a = 2 — ^j 2 . The barriers A A and A~ + also 
coincide at this value of a. For > a > — p — g + 4, neither of the barriers A ++ or 
A intersects the positive quadrant. 

All submodules of S a (X°) are accounted for by the barriers A ±zt and by the 
decomposition (2.8). The interactions between these effects yield several patterns 
of reducibility according to the parities of p and q. The details are quite easy to work 
out. We will summarize the results, with diagrams illustrating typical situations. 

The main point to consider is which of the barriers A ±± affect S a+ (X°) and 
which affect S a ~(X°). For example, suppose (m,n) s (Z+) 2 satisfies A ++ (m, n) = 
while another point (m',n') e (2^+) 2 satisfies A^ (m',n') = 0. Then 

(to + n) - (to' + ri) =a - (2m' -a-q + 2) 
=2(a-m' -l) + q. 

Hence the parity of (to + n) — (m' + n') is the same as the parity of q. Thus if q is 
even, the barriers A ++ and A" 1 both affect the same module of the pair S a± , but 
if q is odd, one of the barriers affects S a+ , and the other affects S a ~ . Similarly, the 
pair of barriers A ++ and A h affects the same module if p is even and different 
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ones if p is odd. Also A H and A~ + affect the same or different modules according 
as p + q is even or odd. 

We describe the various possibilities with a collection of diagrams with commen- 
taries. In all the diagrams below, the arrows on the barriers point toward the region 
of if -types that define a submodule. 

Case OO: p odd, q odd. In this situation, the barriers A' 1 and A h affect the 
same one of the S (X°) and A ++ or A~~ affects the other. For the record, we 
note that A+~ and A~+ affect S ae (X°) when e = (-l) a+1 , while A++ and A 
affect S a£ (X°) when e = {— l) a . We describe the submodule structure for various 
ranges of values of a. 

(i) a > 0. Here the module affected by the barriers A H and A ^ decomposes 
into three components: one irreducible submodule and two irreducible quotients, 
all infinite dimensional, illustrated by Diagram 2.23. 




a + q - 2 



Diagram 2.23 

The other of S a± (X°) decomposes into two components: one finite-dimensional 
submodule and one infinite-dimensional quotient, illustrated by Diagram 2.24. 



a 




a 



Diagram 2.24 

(ii) > a > —[(p + q)/2] + 2. In this range, the structure of the module affected 
by the barriers A A and A h remains qualitatively the same as when a > 0, but 
the other module is irreducible. 
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(iii) a = —[{p+ ?)/2] + 2. For this value of a, the two barriers A A and A h 
coalesce, and there is a submodule consisting of K-types supported on the line 
y = x + (p — q)/2 and two quotient modules consisting of the K-types on one side 
or the other of this line (see Diagram 2.25). 




Diagram 2.25 



(iv) a = —(p + q)/2 + 1. For this value of a, the module affected by A + 
and A~+ (which is S a+ (X°) if (p - q)/2 is odd and S a ~(X ) if (p - q)/2 is even) 
decomposes into the direct sum of two irreducible modules. This is because there 
are no if-types in the region between the two barriers (see Diagram 2.26). 




Diagram 2.26 



This is the only value of a that gives a decomposable module. The other module 
remains irreducible. 

(v) a = —\(p + q)/2\. This situation is dual to case (iii). The constituent with 
a single line of if- types is now a quotient, with barriers trapping the if -types on 
cither side of this line in a submodule. See Diagram 2.27. 
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A+- 




'A~+ 



Diagram 2.27 



Notice that if the module depicted by Diagram 2.26 in case (iv) is S a+ (X°), the 
one affected here is S a ~(X°), and vice versa. The other module remains irreducible. 

(vi) — [(p + q)/2] > a > — (p + q) + 4. Here the picture is dual to case (ii). The 
module affected by and A ^ continues to have three constituents, but now 

it has two submodules and one quotient (see Diagram 2.28). The other module 
remains irreducible. 



-(a + q-2) 




-{a+p-2) 



Diagram 2.28 



(vii) a < — (p + q) + 4. The module affected by A A and A h has qualitatively 
the same structure as in case (vi). The other module now has two constituents: an 
infinite-dimensional subrepresentation and a finite-dimensional quotient (see Dia- 
gram 2.29). 
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-(a + p + q-i) 




(a + p + q-A) 



Diagram 2.29 



Case OE: p odd, q even. In case OE, the barriers A + and A ++ affect one of 
S (X°), and the barriers A h and A affect the other one. For the record, 
A+- and A++ affect S a£ (X°) if e = (-1)°, while A~+ and A affect S ae (X°) if 
e = (— l) a+1 . Here is the description of submodule structure in various ranges of 
the value of a. 

(i) a > 0. Here the module affected by A ++ and A A has three constituents: 
one finite-dimensional subrcprcscntation, one infinite-dimensional quotient, and one 
subquotient in between, as illustrated by Diagram 2.30. 



a 




a 



a + q-2 



Diagram 2.30 



The module affected by A + has two constituents (see Diagram 2.31). 
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Diagram 2.31 



(ii) > a > — (p + q) +4. In this range of a- values, the module affected by A~ + 
retains the same qualitative structure as it has in case (i), but the other module is 
no longer affected by A ++ , so it also consists of only two constituents (see Diagram 
2.32). 




Diagram 2.32 



(hi) a < —{p + q)+A. The situation is dual to that of case (i). The module 
affected by continues qualitatively as in case (ii), while the module affected 

by A h is now also affected by A and has three constituents, including a hnite- 
dimensional quotient (see Diagram 2.33). 
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-(a + p-2) 



Diagram 2.33 

Case EO: p even, q odd. This case, mutatis mutandis, is essentially the same as 
Case OE. 

Case EE: p even, q even. When p and q are both even, all the possible bar- 
riers affect the same one of S a± (X°) (i.e., S ae (X°) if e = (-l) a ), and the other 
is irreducible. The discussion and diagrams below will treat only the reducible 
summand. 

(i) a > 0. Here there are four constituents: one finite-dimensional subrepresen- 
tation, two quotients, and one subquotient in between, as illustrated by Diagram 
2.34. 



a+p-2 
a 




a a + q — 2 



Diagram 2.34 

(ii) > a > — (p + q) + 4. Here, in the range where the barriers A ++ and A 
have no effect, the description of submodules is the same as in cases 00(ii)-(vi). 
The most interesting things happen in the range — [{p+q)/2] + 2 > a > — [(p + q)/2]. 

(iii) a < — (p + g) + 4. The picture is dual to case (i). There is one finite- 
dimensional quotient, two infinite-dimensional subrepresentations, and one con- 
stituent in between (see Diagram 2.35). 
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-(a + q-2) 
-(a + p + q-A) 



Diagram 2.35 

Remarks, (a) Although there are considerable variations in the submodule struc- 
ture of S a (X°) according to the parities of p and q, there is always a total of five 
constituents for a > 0, one of which is finite dimensional. 

(b) The extra complexity when p + q is even of the submodule structure in the 
range — [(p + q)/2] + 2 > a > —[(p + q)/2 will acquire additional significance in §3 
where unitarity is investigated. 

(c) Although we have only described the constituents of S a (X°) as "finite dimen- 
sional" or "infinite dimensional", the Diagrams 2.23, 2.28, 2.34, and 2.35 certainly 
suggest that the constituent "in the middle" is smaller than the constituents "in the 
wings" . This difference in size could be made precise in terms of Gelfand-Kirillov 
dimension [Vol] or A-spectrum [Ho2]. 

Degenerate cases p > q = 1 . We will now indicate how the case of q = 1 differs from 
the picture developed above and also make some supplementary remarks about the 
case q = 2, which is also somewhat exceptional. 

When q = 1, the light cone X° is not connected. We can divide it into two 
relatively open and closed pieces X 0± , called nappes, defined by 

X 0± = {(x,y)eX°\y = ±\x\}. 

(Note that y is just a real number when q = 1.) The two nappes A 0± of the 
light cone are each stabilized by a subgroup + (p, 1) of index 2 in 0(p, 1). We 
see that any element of S a± (X°) is determined by its restriction to A 0+ . Fur- 
thermore the function sign on A" , which takes the values ±1 on A 0± , is clearly 
invariant under + (p, 1) and is an eigenfunction of 0(p, 1). Evidently multiplying 
by sign gives an + (p, l)-module isomorphism between the S a± (X°). Thus when 
q = 1, analyzing S a (X°) as an 0(p, 1) module is essentially the same as analyz- 
ing S a+ (X°) ~ S a (X°+) as an 0+(p, 1) module. We will do this, following an 
abbreviated version of our agenda for the case of 0(p, q), p, q > 1. 
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The map 

(2.36) (3 : X°+ - S^ 1 x R*, 0(x, y) = (x/y, y), 

where x € M. p — { } and y — r^x) 1 / 2 e R*, is a diffeomorphism. The maximal 
compact subgroup of + (p, 1) is 0(p), which acts transitively on the set {(x, 1) G 
X 0+ } ~ S p_1 . We can define embeddings 

(2.37) j a>m = j a : H m (W) - S a (X°+), j a (/i)(z,y) = ft^Jy-" 1 . 

Note that since y > on X 0+ , complex powers of y make sense. From decomposi- 
tion (2.36) and the theory of spherical harmonics, we know that the decomposition 
of S a (X°+) as an 0(p) module is 

S a (X°+) ~ ^ . 7a (H m (IR p )). 

m>0 

To complete the picture we want to compute the action of p (see formulas (2.6)) 
on the if -types j a (H m (W j). The analogue of Lemma 2.3 is 

= (a- m^V"™- 1 + {a + m+p- 2)hjy a - m+1 
= (a- m)T+(h)y a - m - 1 + {a + m + p - 2)Tr (h)y a - m+1 , 



where 



dh 

h+=x j h-(2m+p-2)- 1 — rl 



dx 3 p ' 

hj = (2m+p-2)- 1 — , heH m (R p ). 
oxj 

We note that hj is harmonic, while fij is the projection of Xjh into H m+1 
The principles behind this calculation are discussed in §6.1. It follows easily that 
S a (X 0+ ) is always irreducible except when a is an integer, with either a > or 
a < —p + 1, in which case S a (X 0+ ) has two constituents, one of which is finite 
dimensional. In what we hope will be a self-evident analogy with the situation for 
q > 1, we illustrate this by Diagram 2.38. 



a > 



a < -p + 1 •- 



-(a + p-2) 

f— 



Diagram 2.38 

For — p + 1 < a < 0, the representations S a (X 0+ ) are irreducible. 
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Although the general picture presented in Diagrams 2.23-2.35 is valid when 
q = 2, this case is also somewhat exceptional. Although the H n (R 2 ) are nonzero 
for all n > 1, their dimensions, instead of growing with n, remain fixed at 2 (for 
n = 0, H n (M. q ) is the trivial representation for all q and has dimension 1). More 
significantly, whereas the representations H n (M. q ), n > 1, q > 3, of 0(q) remain 
irreducible when restricted to the special orthogonal group SO(g), the representa- 
tions H n (M. 2 ), n > 1, decompose into two one-dimensional representations when 
restricted to SO(2), which is isomorphic to T, the unit circle. This suggests that 
one consider varying the picture presented above by restricting the representations 
S a (X°) to the identity component 0°(p, q) of 0(p, q). For p, q > 3 this has no 
effect. When q = 2 (and p > 2), however, the pictures presented above for 0(p, 2) 
must be "folded out" along the x-axis to yield pictures appropriate for 0°(p, q). 
Precisely the Diagrams 2.30, 2.31, and 2.33 now should look as in Diagrams 2.39, 
2.40, and 2.41 respectively. 



a 




'A+ 



—a 



Diagram 2.39 



a + p-2 





Diagram 2.40 
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Diagram 2.41 

We observe that in Diagram 2.22, the gap between the points where A ++ and 
A A intersect the x-axis is q — 2, so when q = 2, these points coincide, and so when 
reflected across the x-axis, these barriers make crossing straight lines. Similar 
remarks apply to A~~ and A h . 

If p is even, then the relevant diagrams are 2.34 and 2.35. The modified version 
of Diagram 2.34 is Diagram 2.42. 



a + p-2 
a 



—a 

-(a + p-2) 




Diagram 2.42 

Thus in this case, we have six constituents in all, with Hasse diagram [Sta] as 
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illustrated by Diagram 2.43. 




Diagram 2.43 



We omit the diagram corresponding to Diagram 2.35, which would be dual to 
Diagram 2.42. The main effect of the restriction is that representations that have 
no if -types along the x-axis, that is, that contain no fixed vectors for 0(2), break 
up into two summands symmetrically placed across the x-axis. However, represen- 
tations whose K-type region includes part of the x-axis remain irreducible. Rein- 
forcing this splitting is the fact that when q = 2, the barriers A ++ and A A (and 
the barriers A and A '") intersect on the x-axis, leaving no gap for if -types 
of the intermediate constituent in Diagram 2.30. Hence this constituent becomes 
two on restriction to 0°(p, 2). The representations of 0°(p, 2) whose if -types are 
restricted to only one side of the x-axis are of the type known as holomorphic or 
highest weight modules [EHW, Br]. 

If p — 2 and q = 2, then Diagram 2.42 should be further folded out along the 
y-axis. Since 0(2,2) is just isogenous to SL(2,R) x SL(2,R), we will not dwell on 
this example. 



In this section we will investigate which of the subquotients of the S a (X°) de- 
scribed in §2 can define unitary representations, that is, possess an 0(p, ^-invariant 
positive-definite Hermitian form. As with the calculations of submodule structure, 
the results are primarily based on the formulas of Lemma 2.3 for the action of 
P C o(p,q). 

Since the K -types j a ('H m (M p )<E)H n (M q )) are irreducible modules for the compact 
group 0(p) x 0(q), each one admits an 0(p) x O (^-invariant Hermitian inner 
product, and this is unique up to multiples. For this discussion we will fix an inner 
product (•, •) m n on each space H m (M. p ) ® H n (R q ) by the geometric recipe 



where <t> 3 G H m (W)®H n (R q ) and dz indicates the 0(p) x 0(g)-invariant probability 
measure on x § 9_1 . We will push forward this inner product by j a to obtain 
an inner product on the X-type j a (H m (W) ® H n (W)). Wc call the collection of 
inner products so obtained the standard inner products on the i\T-types. 

If (•, •} is an 0(p, ^-invariant Hermitian form on S a (X a ) (or on some constituent 
of S a (X )), the restriction of (•, •) to any if-type must be a multiple of the standard 



3. Unitarity 



(3.1) 
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inner product on that K-type: 

(3-2) 0a(/l),ja(/ 2 )) (/l,/2> m ,„, 

where fj e H m (M. p ) <g> W"(R 9 ) and c m . n is an appropriate real number, which must 
be positive if (•, •) is to be positive definite. Furthermore the X-types must be 
mutually orthogonal, so that (•, •} is determined by the numbers c m>n . 

According to an early theorem of Harish-Chandra [HC2] , a positive-definite inner 
product (•, •) on S a (X°) will be 0(p, q) invariant if and only if the operators from 
o(p,q) are formally skew adjoint with respect to (•,•). Equation (3.2) guarantees 
this will be so for operators from o(p)(Bo(q). Thus to determine invariance of (•, •) we 
just need to check skew-adjointness of the operators from the subspace p C o(p,q). 
From formulas (2.18) and (3.2), we see that this amounts to the equations 

(a) =(a - m - n)c m+hn+1 (T++(z ® 0), <f>') m+1>n+1 

+ (a + m + n + p + q- 2)c ro ,„(t/>, T~+^ n+1 (z ® 4>')) m>n , 

(b) 0=(a-mtnt?-WiO^)^'U,n-i 

+ (a + m-n + p)c m ,n(<t>, T m+l,n-l( Z ® Om,n' 

where z e p, <j> G U m {W) ® H n (R q ), <f>' e H m+1 (W) ® H n+1 (R q ), and <f>" G 
H m+1 {W) ® W* -1 ^)- Here a means the complex conjugate of a. 

Equations (3.3) can be substantially simplified by an elementary observation, 
which will be justified in §6. 

Lemma 3.1. When Re a = ~[(p + q)/2] + I, that is, for a = — [(p + q)/2] + 1 + is, 
s E K, the Hermitian inner product which equals the standard inner product on 
each K-type, is 0(p,q) invariant. 

Remark. More generally, one can show that if <\>\ in formula (3.1) is taken to be- 
long to S a (X°) and cj> 2 is taken from S a '(X°), where a + a 1 = 2 - (p + q), then 
(3.1) defines an 0(p, ^-invariant pairing between S a (X°) and S a (X°), which are 
thus canonically mutual Hermitian duals. This duality explains the symmetry in 
structure among the S a (X°), as discussed in §2. 

Proof. Sec §6.1. □ 

If we take a — — [(p + q)/2] + 1 in formula (3.3), we see that Lemma 3.1 implies 

(a) <T++(*®0),^) m+li „ +1 = (<f>, T~+ l n+1 {z ® 0'))m,n' 

(b) (T+-(^),0") ro+w = <<^|i,„-i(*®<£")> ro ,„ 

for all possible choices of z, </>, <j>' , <f>" as in (3.3). Given these relations, a reinspec- 
tion of formulas (3.3) shows they are equivalent to the purely numerical equations 

; m+i,n+i + (a + m + n + p + q- 2)c m ^ n = 0, 
- 2)c m+ i jn _i + (a + m - n + p)c rn . n = 
or 




(3.4) 



A ++ (m, n)c m+Ml+i + A (m + 1, n + l)c m ,„ = 0, 
A+~(m,n)c m+ i ;n _i + A-+(m+ l,n- l)c TOj „ = 0. 
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Let us explore the implications of equations (3.4). Consider first the case when 
the S a± (X°) arc irreducible. Then the quotient 



m + n 



a+m+n+p+q— 2 
must at least be real for all possible choices of to and n. Setting 

a = a-\ 1, 

we can rewrite this ratio as 

to + n + 5 — a (to + n + 5 — Re a) — i Im a 
m + n + 5 + a (m + n + S + Rea) — ilmo' 

where 



P- 



1. 



It is easy to convince oneself that the ratio (3.5) can be real for large to, n only if 
either Re 5, = or Ima = 0. The case of Re a — is covered by Lemma 3.1. We 
will call the set of a with Re a = the unitary axis. If Ima = Ima ^ 0, we know 
(see Proposition 2.4) that S a± (X°) is irreducible; hence for it to be Hermitian, 
expression (3.5) would have to be real for all to, n. This leads us to the following 
conclusion. 

Lemma 3.2. In order for S a (X°) to have any Hermitian constituents, either a 
must be on the unitary axis, or a must be real. Conversely, for a on the unitary 
axis, S a (X°) is unitary, and for a real, all components of S a (X°) are Hermitian 
(i.e., they possess an invariant Hermitian form). 

The final assertion follows from the observation that if a is real then we can 
start at a given K-type (mo, no) in any constituent of S a (X°), set c m0j „ = 1, and 
use equations (3.4) to determine c m .„ for other i^-types in that constituent. It 
should be checked that this procedure is consistent, i.e., that it results in a well- 
defined c m ,n on each K-type. For this, it suffices to check that the transitions 
around squares, like (to, n) — ► (to + l,n + 1) — ► (to + 2, n) and (m,n) — ► 
(to + 1, n — 1) — ► (to + 2, n), predict the same value for the ratio of the start and 
finish (here c m+ 2, n /c m ,„). This is a short computation. Here, as in the analysis of 
irreducibility, it is germane to note that the transition coefficients A ++ and A~~ 
are unchanged by the transitions (to, n) — ► (to, n) ± (1,-1); likewise A A and 
A h are unchanged under (to, n) — ► (m, n) ± (1, 1). 

Our goal is now to investigate for which constituents of S a (X°) the process 
described in the previous paragraph will yield positive values for all c mj „ in the 
constituent. Looking again at equations (3.4), we can easily see and formulate the 
following rule. 

Criterion 1. Let a be real. A given constituent of S a (X°) is unitary if and only 
if the ratios 

r „ „n / • \ A ++ (m,n) , .... A^ (m.n) 

( 3 - 6 ) (0 , — ,„ h ' i\ and (") 



-(m + l,ra+l) A-+(m+l,n-l) 
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are negative for all pairs of if -types (to, n) and (to + 1, n + 1) or (to, n) and (to + 
l,n — 1) contained in the constituent. 

We proceed to investigate the nature of this criterion in order to obtain an 
explicit list of the unitary constituents. 

Consider again our graphical representation of the if-types as integer points in 
the positive quadrant, together with barrier lines corresponding to the zeroes of the 
linear functions A ±=t . We illustrate the situation for a typical a > in Diagram 
3.7. 




Diagram 3.7 



In this diagram we have indicated, for each transition coefficient, the side of 
its barrier where it is positive and the side where it is negative. We see that, in 
the positions illustrated, both A H and A h are positive in the region between 
their barriers and likewise for A ++ and A . When a < and the A^ barrier 
is above the A h barrier, both A^ and A h are negative in the region between 
barriers. In general, we may say that A H and A h have the same sign in the 
region between barriers, and have opposite signs in the regions outside the barriers, 
similarly for A ++ and A . Thus, if (to, n) and (m + 1, n— 1) both lie between the 
A H and A h barriers, or more precisely if (to, n) lies above the A A barrier and 
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(to + 1, n—1) lies below the A h barrier (or vice versa), then the ratio (3.6) (ii) is 
positive and unitarity cannot hold. On the other hand, if (to, n) lies above the A A 
barrier and (m+ 1, n — 1) also lies above the A h barrier (or both lie below), then 
the transition from (m, n) to (m + 1, n — 1) is consistent with unitarity. Similar 
remarks apply to the pair A ++ , A~~. 

We can apply these considerations to determine the unitarity of constituents of 
the S a (X°). As for the discussion of submodule structure in §2, it is convenient to 
distinguish cases according to the parities of p and q. Before beginning we note that, 
using the notation a as in formula (3.5), the same argument that establishes Lemma 
3.1 shows that the modules S a (X°) and S~ a (X°) are mutual Hermitian duals. This 
will be shown in §6.1. It follows that in considering questions of unitarity, it will 
suffice to take a > 0. 



Case OO: p odd, q odd. From the unitary axis (a = 0) until a = [(p + Q)/2] — 1 
(i.e., a = 0), neither of the barriers A ++ or A intersects the positive quadrant. 
Hence unitarity depends only on the barriers A^ and A h . Their position when 
a = is illustrated in Diagram 2.26. Let e denote the choice of ± such that 
S aea (X°) is reducible at a — 0, and let E\ denote the other choice. For the record, 
we note that £o is + if (p — q)/2 is odd and is — if (p — q)/2 is even. 

At a = none of the K-types of S as °(X°) lie between the barriers A^ and A 
which are located on the two lines L±i defined by equations y — x= [(p — q)/2] ± 1; 
however, there are i-T-types lying on these two lines. As a increases from 0, the 
barrier A A moves to the right and A~ + moves left. Hence, for < a < 2, 
transitions between L\ and L_i cross both barriers and so violate unitarity. For 
a > 2, transitions between L\ and L_i cross neither barrier and so again violate 
unitarity. Hence the full module S ae °(X°) is never unitary when a > 0, and at 
points of reducibility the constituent with K-types between the barriers cannot be 
unitary. The other two constituents, however, are easily seen to be unitary, since 
all their transitions take place on the same side of both barriers. 

The module S a£l (X°) is irreducible at a = and remains so for < a < 1. 
For this range of a, the only K-types between the barriers are those on the line Lq 
defined by y — x = (p — q)/2. One checks that transitions in either direction from 
Lq are consistent with unitarity. At a = 1 the barriers coincide on Lo (see Diagram 
2.25), and the if -types on this line span a constituent of S a€1 (X°). There are no 
transitions transverse to the barriers from Lq, so this constituent is unitary; likewise 
the other constituents are unitary, so all three are unitary. For a > 1 the line Lq is 
on the opposite side of the barriers A" 1 and A from where it was when a < 1, 
so the transitions from Lo violate unitarity. Hence unitarity for S a£l (X°) stops at 
o=l. Also, for points of reducibility beyond a = 1, the same reasoning applies as 
toS* a£l (A°). 

In the range a > 0, when there may be a finite-dimensional component, one 
easily checks that the trivial representation at a = is unitary, but no other finite- 
dimensional representations are so. 

In sum, the unitarity properties of the 5 ,o± (X ) in this case are summed up in 
Diagrams 3.8 and 3.9. For Diagram 3.9, we note that (a) and (b) depict S ae (X°) 
when e = (-l) a+1 , while (c) and (d) depict S ae (X a ) when e = (-l) a . 
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S a£0 (X°) : 



reducible point 
at a = 



unitary axis 
Re a = 



S aEl (X°) : 



unitary axis 
Re a = 



DIAGRAM 3.8. Parameter values for unitarity of the full modules S a± (X°). 




(a) at a = 1 (b) a e Z, a > 1 



nonunitary 




(c) a = (d a e Z, a > 0) 



Diagram 3.9. Unitarity at points of reducibility. 

Case OE: p odd, g even. In this case, when 5 = 0, the barriers do not pass 
through integral points, so the representations S~ S± (X°) are irreducible (recall 
that S = [(p + q)/2] — 1). Each has if -types between the barriers: one on the 
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line L X j2 defined by y — x = \{p — q)/2] + 1 and the other on the line L_ 1 / 2l 
defined by y — x = [(p — q)/2] — i (see Diagram 3.10). As a increases from 0, 
the barriers approach these lines and coincide with them for a — ^. One checks 
that for < a < \ transitions from these lines is consistent with unitarity, but 
as a passes |, the barriers move past the lines L±i/2, and then transition from 
these lines does violate unitarity. Hence for a > 5 , the full modules S a± (X a ) can 
no longer be unitary. At points of reducibility for a > \, one can check that the 
constituent with X-types to the right of and the one with if-types to the left 
of A ^ is unitary. Again the only unitary finite-dimensional unitary constituent is 
the trivial representation. Diagrams 3.11 and 3.12 illustrate the main conclusions. 
For Diagram 3.12, we note that A++ and A+~ affect S as (X°) if e = (-l) a , while 
A~+ and A affect S ae (X°) if e = (-l) a+1 . 




Diagram 3.10. a = 0. 



unitary axis 
Re a = 



5G(-U) 



DIAGRAM 3.11. Parameter values of unitarity of the full modules S a± (X°). 
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unitary 




(p-g-2 




(a) a = \ 





a — 



(p-g+2) 



(b) 5 e \ + Z, o > \ 



(c) a e \ + Z, a > 5, a < 




unitary: 
trivial 

representation 




nonunitary 




unitary 



(d) a = 



(e) 5 e | + Z, a > 



Diagram 3.12. Unitarity of constituents at points of reducibility. 

Case EO: p even, q odd. This case is essentially the same as case OE. We will 
not give the details. 



Case EE: p even, q even. Here the discussion is essentially the same as for Case 
00, except that, when a > 0, at a point of reducibility, three barriers affect the 
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same S a± (X°), and the other one is irreducible. We draw the pictures illustrating 
this situation (see Diagrams 3.13(a) and (b)). 




a a+q-2 
(b) a £ Z, a > 

Diagram 3.13 



DEGENERATE PRINCIPAL SERIES 



31 



Remarks, (a) It is interesting to note that both summands S (X°) of S a (X°) 
"see" all the barriers via their Hermitian forms, whether or not these barriers cause 
reducibility in one summand or the other. In fact, they see the barriers even when 
the barriers do not pass through integral points at all. 

(b) The unitary representation with AT-types distributed along a single line which 
occurs at a = 1 when p + q is even [see Diagram 3.9(a)] is particularly interesting. 
It is paired with the trivial representation in the local correspondence for the dual 
pair (0(p, q), SL(2, R)). It is the smallest possible nontrivial unitary representation 
of 0(p, q) and may be regarded as a "quantization of the minimal nilpotent orbit" 
in the argot of geometric quantization [EPW] . It turns up in physics in connection 
with the quantum Coulomb problem [Ab, AFR, DGN, Fr, On]. 

(c) The representations S a+ (X°) are "spherical" representations, meaning that 
they contain the trivial representation of the maximal compact subgroup, and the 
^"(A ) are nonsphcrical representations. Notice that, for each of these families or 
"series" of representations, the only point on the unitary axis that can be reducible 
is the point on the real axis, namely, a = 0. When p + q is even, exactly one of the 
two series is reducible at a = 0, while when p + q is odd neither series is. The group 
SL(2,R) has two principal series of representations, analogous to the S a± (X°): one 
spherical series and one nonsphcrical. The spherical series is irreducible on the real 
axis, and the nonsphcrical series is reducible. Also, as was well known and has 
been shown again here, the unitary spherical principal series for 0(n, 1) is always 
irreducible. These examples perhaps created some expectation that spherical series 
were more likely to be irreducible than other series. In our example, however, when 
p + q is even, either the spherical or the nonspherical series may be reducible at 
a = 0, according to the parity of (p — q)/2, while when p + q is odd, neither series 
is reducible. 

We note also that when a is a positive integer, the trivial A-type is always 
contained in the central strip where nonunitarity reigns. Thus outside the unitary 
axis and its immediate neighborhood (the complementary series), there are only 
finitely many unitary spherical representations. 



Degenerate cases p > q = 1. In closing, we record the analogous pictures for 0(p, 1). 



DIAGRAM 3.14. Parameter values for unitarity of the full modules S' a± (A ). 



unitary axis 



Rea = -(H f i) 



a = 




ae(-(p-l),0) 
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unitary: . unitary. 



trivial representation 
(a) a = 



nonunitary 1 unitary 



(b) a E Z, a > 



Diagram 3.15. Unitarity at points of reducibility. 



The reason that the unitary representations can live so far off the unitary axis is 
that since the if-types now live along a single line, the A A and A h barriers no 
longer affect unitarity — only the barriers which cut off finite-dimensional represen- 
tations, and these do not come into play until the finite-dimensional representations 
begin appearing at a = 0. 



4. The U(p, o)-module structure of homogeneous functions 

ON LIGHT CONES 

4.1. The U(p, q) modules S a '"(X°). Wc now wish to adapt the analysis given 
for 0(p, g) in §§2 and 3 to the indefinite unitary groups V(p, q). 

Let C p+q = C p © C q denote complex (p + q)-dimensional space, parametrized by 
tuples (z, w), with z — {zi,z 2 , ... ,z p ) in C p and w = (wi,w 2 , . . . , w q ) in C q . Wc 
endow C p+q with the Hcrmitian form 



of signature (p, q). Let U(p, q) C GL(p + q, C) denote the group of linear isometrics 
of the form (•, ^p.q. Observe that the subgroup U(p) x U(g), which preserves the 
z variables and the w variables, is a compact subgroup of U(p, q). (In fact, it is a 
maximal compact subgroup.) Let 




(4.1) 



X° = {r,eO+*-{0}\(r,,Ti) p , q = 0} 

= {(z,w)eC p+q -{0}\(z,z) p = (w,w) q } 
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be the light cone or variety of null vectors. By Witt's Theorem [Ja] or by elementary 
means, one can see that U(p, q) acts transitively on X°. We wish to study the action 
of U(p, q) on appropriate spaces of functions on X°. 

If we regard C p+9 as M. 2 p+ 2 i by means of coordinates xi , yi , where 

zj = x 2 j-i+ix 2 j, l<j<P, 
w k = Vik-\ +iV2k, l<k<q, 

then the real part of the form becomes the indefinite inner product of 

signature (2p,2q), the light cone (4.1) is the same as the light cone for R 2p+2q , 
and U(p, q) is identified to a subgroup of 0(2p, 2q). Thus we are led to consider 
the action of XJ(p, q) on the spaces S a± (X°) defined by restriction of the action 
of 0(2p, 2q). These spaces, however, can clearly never be irreducible for U(p, q), 
because XJ(p,q) commutes not simply with R x , the real scalar dilation operators 
used to define the S a± (X°), but with C x , the complex scalar dilations. (Indeed, 
V(p,q) may be characterized as the subgroup of 0(2p,2q), which commutes with 
C x .) Thus it is appropriate to break up the spaces S a (X°) into subspaces that are 
homogeneous for the action of C x (i.e., eigenspaces for C x ). We observe 

(4.3) C x ~R X -T = {\t\-t/\t\ teC x }. 

Here T = {s G C x | \s\ — 1} is the usual unit circle. Thus a character of C x has 
the form 

(4.4) ij){t) = \t\ a (t/\t\) n , a e C x , neZ. 

Since \t\ = (tt) 1 ' 2 where i is the complex conjugate of t, we can formally write 

(4.5) iP(t) = ip a - (3 {t) = t a t , 

with a = (a + n)/2 and (i = (a — n)/2 or a = a + [3, and n = a — [3. We see that a 
pair (a, (3) can occur in (4.5) if and only if a — (3 is an integer. For all such pairs, we 
define S a ^{X°) C S a+f:! {X ) to be the ip a ^ eigenspace for C x . We clearly have a 
decomposition 

(4.6) S a (X°) = S a -P(X°). 

a+P=a 
a~f3£Z 

Further each summand in equation (4.6) is obviously invariant under the group 
U(p,q). We will study the representations of XJ(p,q) defined by the S a ^(X°). Wc 
remark that they can be interpreted as induced representations, again "degenerate 
principal series", just as the S a± (X°) were interpreted as induced representations 
of 0(2p,2q) (see formulas (2.9)^(2.11)). When n = (or when a — [3 and only 
then), the representations S a - j3 (X°) will be spherical, that is, will contain a vector 
invariant under the maximal compact subgroup U(p) x \J(q) C U(p, q). When q = 1 
and a = (3, we have the spherical principal series of U(p, 1). 

To understand the U(p, q)- module structure of S a ' l3 (X°), we proceed as we did 
for the S a (X°): First we describe the action of the maximal compact subgroup 
U(p) x U(g) and then we see how the noncompact part of the Lie algebra u(p, q) of 
V(p,q) moves around the U(p) x U (^-isotypic subspaces. 



34 



R. E. HOWE AND E. C. TAN 



4.2. ^-Structure of S a >P(X°). Under the inclusion of XJ(p,q) in 0(2p,2g), it is 
clear that U(p) x U(q) C 0(2p) x 0(2g). Since the 0(2p) x O(2g)-modulc structure 
of the S a (X°) is described in terms of spherical harmonics (see formulas (2.12)- 
(2.16)), we are led to consider the structure of the space H m {C p ) ~ H m (M. 2p ) as a 
U(p) module. 

Consider the algebra "Pr(C p ) ~ 'P(IR 2p ) of polynomials on C p considered as a 
real vector space. We may choose the complex coordinates z\, z 2 , ■ ■ ■ ,z p and their 
complex conjugates ~zi,Z2, ■ ■■ ,~zp~ as generators for Pr(C p ). For integers a, f3, let 
■p a '^(C p ) be the space of polynomials that are homogeneous of degree a in the zj 
and homogeneous of degree (3 in the ~zj. This notation is consistent with that of 
§4.1, in the sense that V a, @(C p ) is the ip a, @ eigenspace for the action of C x on 
"Pr(C p ) by scalar dilations. Clearly we have a decomposition 

(4.7) V^{<C P )~ Yl ^ Q,/3 (C P ) 

of M-homogeneous polynomials of degree m into bihomogeneous polynomials of 
appropriate bi-degrees. 

The Laplacian on C p = M. 2p can be written 

A = V— = if 32 
f-f dx 2 4-f 9z 7 az-' 

From this formula it is clear that A maps V a ^(C p ) to pa-i^-i^p^ It f u ows 
that the harmonic polynomials have a decomposition analogous to (4.7), namely, 

(4.8) n m (c p )= J2 n m {<c p )c]V a - f3 {c p )= J2 ^ a,/3 (c p ), 

where the second equation serves to define the H Q '^(C P ). Since U(p) C 0(2p) 
commutes with C x , the spaces H a 'P(C p ) will be invariant under the action of U(p) 
on H m (C p ). Hence they define representations of U(p). It is known (see [Zh]; this is 
also a special case of the general results of [Ho3]) that the H a 'P(C p ) are irreducible 
and mutually inequivalent U(p) modules. We note that, in particular, the spaces 
H a 'P(C p ) are eigenspaces for the group C x (~l U(n) ~ T of scalar unitary matrices; 
the eigencharacter by which T acts on TL a ^{£. p ) is 

(4.9) x a -0 ■ * z a ~ f) - 

If p > 2, then H a 'P(C p ) is nonzero for all a, (3 > 0; however, when p = 1, only 
H a '°(C) = Cz a and H°' /3 (C) = Cz 13 are nonzero. This degeneracy in the H aJ3 {C p ) 
when p = 1 leads to some interesting phenomena, which will be detailed later in 
the paper (see §4.5). 

Next consider the action of U(p) x U(g) on the algebra Pr(C p+<? ). Taking the 
tensor product of equation (4.8) with its analog for C q gives us a decomposition 

(4.10) H m {C p )®H n (C q ) ~ ^ ?< mi ' m2 (C p ) (8»H ni ' n2 (C 9 ). 

mi+m2= m 
ni +n 2 — n 
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The action of the scalar unitary matrices on the summand H mi ' m2 {C p )®H ni ' ri2 (C«) 
is via the character \r where r = mi — m 2 + n\ — n 2 [sec definition (4.9)]. Using 
this, we can easily see that the maps j a , m ,n of formula (2.15) split into maps 

(4.11) j Ql/3 , roi , ro2 , ni ,„ 2 =j a ,f, : W mi < m2 (<C p )®W ni <" 2 (C«) -+S a 'f>(X°), 
where 

a + [3 = a, a — [3 = mi — m 2 + n\ — n 2 - 
In this connection we observe that 

(4.12) r 2 p = |^ 2 = £^ 

so that r| p is in 7 3l ' 1 (C p ) and multiplication by any power of r\ p increases both 
components of bidegree equally and so does not affect their difference. 

Combining formulas (2.16), (4.6), (4.10), and (4.11) yields the decomposition 

S a 'P{X°)= {i«+f){H m {€ p )®n n (<£ q ))C\S a ^(X Q )) 

m.n>0 



= E ja,/j(« mi ' ma (C P )®W ni ' na (C«)) 



(4.13) 



m,n>0 mi ,m2,ni ,n2>0 
mi+m2=m 
ni+ri2=n 
mi — m2+rii — 712 = a — /3 



^ ja, /3 (?r ,iima (c , ')®?r i,n2 (c 9 )) 



mi,m2,ni,?i2>0 
mi — m.2+ni — 112 — a — /3 

of S a ^(X°) into irreducible U(p) x U(<?) modules. 

From the final form of the decomposition, we see that the U(p) x U(g) compo- 
nents of ^"'^(X ) form a three-parameter slice out of the full four-parameter family 
of U(p) x U(g) representations that appear in Pk(C p+9 ). We may refer to these 
irreducible U(p) x XJ(q) modules as "if -types". The middle form of the decomposi- 
tion emphasizes the relation of these U(p) x \J(q) modules and the 0(2p) x 0(2q) 
modules of the first form of the decomposition. We see from formula (4.10) that 
the U(p) x V(q) submodules in H m (C p ) x H n (C q ) can be parametrized by the pair 
(mi, ni), which varies in the integral points in a rectangle (see Diagram 4.14). 

(mi,m) = (m,n) 



I I 
I I 
I I 
I I 

* __ ~ " ~ *(mi.m) = (0.0) 



Diagram 4.14 
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The subset of points such that mi — m 2 + ri\ — n 2 = a — (3 forms a diagonal line 
inside this rectangle (see Diagram 4.15). 




Diagram 4.15 

Here we should observe that since 

mi - m 2 + m - n 2 = mi + m 2 + n\ + n 2 - 2(m 2 + n 2 ) 
= m + n- 2(m 2 + n 2 ), 

the inner sum in the middle line of formula (4.13) will be empty unless 

(i) m + n > \a — 

(ii) m + n = a — [3 mod 2. 

When q = 1, there are further restrictions, described in §4.5. For now, we assume 
P,Q > 1- 

To exploit the relation between the situation under study and the results of §§2 
and 3, we will represent formulas (4.13) graphically as follows. We think of the 
points (m, n) for which the inner sum in the middle line of (4.13) is non-empty 
as being integral points in the plane. This gives us all points in a coset of the 
"diamond lattice" in the positive quadrant and lying above the line x + y = \a — f3\, 
as illustrated by Diagram 4.16. 




Diagram 4.16 



We further think of the "line" of U(p) x U(q) modules in the inner sum of the 
middle line of (4.13) as points in a "fiber" lying over the point (m, n) in Diagram 
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4.16. We call the region 

{(x,y) \ x>0, y>0, x + y> \a - 0\) 

the K-type region. 

4.3. Action of the noncompact part of u(p, q) on S a '"(X°). Consider now 
how the noncompact part of the Lie algebra u(j>, q) acts on the U(p) x XJ(q) compo- 
nents, as described in the preceding section. Since u(p,q) C o(2p, 2q), it must act 
compatibly with Diagram 2.19, so the U(p) x U(q) modules in the fiber over (to, n) 
must be taken to the fibers over (mil, nil); but since each fiber consists of 
several U(p) x U(g) modules, we must calculate how the u(p, q) action affects the 
various points in each fiber. To formulate this we observe that the complexified Lie 
algebra u(p, q)c has a decomposition 

(4.17) u(p, g) c - (u(p) © u(g)) c © P + © P", 

where p ± are the eigenspaces for the adjoint action of the center of u(p) © u(q). 
Concretely we have 

p+=span \S ij = z +w 

(4 - 18) L 9 - d \ 

p-=span l [ T ij = Wj —+z i — j. 

The following lemma is a refinement to the current situation for Lemma 2.3. We 
leave the calculations justifying it to §6.2. 

Lemma 4.1. There exist nonzero maps 

r.1001 cO-101 clO-10 cO-l-10 

J mi,m2,fii,n2' ^7771,7772,771,77.2' mi ,7712 ,ni ,712 ' mi, 7772,771,772 ' 



y.0110 J.-1010 /tiOIO-1 ji-100-1 

7771,7772 , 771,772 ' 7771,7772,711,712' 7771,7712,711,772' 7771,7772,771,712 ' 



where 



S^XX,n 2 ■■ P + ® (W 7 " 1 '™ 2 ^) © H"-" 2 (C)) 

y 'J-jJUl+El ,m 2 +£2 (C?) (g) -^"1+7/1, "2 +7/2 (C?) 

rSX,n2 : P" ® (^ mi ' m2 (c p ) © w^c)) 

y j^m 1 +ei ,m 2 +E 2 ((£P) ^"1 +^1 ,"2+7/2 (C?) 

independent of a, (3, such that the action of z G p + and z G p~ [see (4.18)] on i/ie 
if -type ia,/3(H mi,m2 (C p ) © H ni ^ 2 (C«)) is described by the formula 

p(z + Z)j a ,p((t>) 

= A++(m,n)j a ,p {S^ m2inun2 (z © cf>) + T™ ai „ liBj (z © 0)} 
+ A-+(m,n)j ai/9 {^T^ 2i „ 1;n2 (^ © 0) + T"*™ nii „ 2 (ii © 0)} 
+ ^+-(m,n)j aj/3 {^°-i° 2i „ 1;n2 (^ © 0) + T™-l nitn2 (z © 0)} 

+ A— (TO, n)j a>P {^-^ 2 10 ni ,„ 2 (^ © 0) + T , TO 1 1 1 TO 2 ",ni,7l2 ® ' 
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where m = m x + m 2 , n = m + n 2 , <j> £ H mi ' m2 (C p ) <g> H ni > n *(C q ), and A ±=b = 
^2p2qa+0 * s as defined in (2.20). 

Proof. See §6.2. □ 

The action described in this lemma can be pictured as follows. Imagine the fibers 
over (m,n) and (m + l,n+ 1) as lines of points, as suggested by Diagram 4.15. 
Then the effect of the S and T operators of Lemma 4.1 can be illustrated as in 
Diagram 4.19. 



(mi ,m2.ni,n 2 ) (mi + 1,7712 — l,7li — l,n 2 + l) 




(mi, 7712 + 1,711 + 1, "2) (mi + 1,7712, 711 ,712 + 1) (mi +2, 7712 — 1,7»1 — 1,712+2) 



Diagram 4.19 

From Diagram 4.19, we see that, providing the transition coefficients A ++ , A 
are nonzero at (m,n) and (m +1, n + 1) respectively, that is, providing that the 
fibers over (m, n) and (m + 1, n + 1) belong to the same o(2p, 2g) constituent of 
S a+ P(X°), application of an element of p + followed by another from p + moves a 
given U(p) x U(</) module to its neighbor in the same fiber. Computations in §6.2 
will show that by successive applications of (p + ) 2 [or (p~) 2 ] we can move from any 
point in a fiber to any other. A similar analysis applies to transitions between 
(m, n) and (to + 1, n — I). This implies the result: The U(p, q)- module structure 
of S a - j3 (X°) is controlled by the 0(2p, 2(7)-module structure. 

Theorem 4.2. Assumep, q > 2. The image of S a '@ (X°) in any 0(2p, 2q) -irreducible 
constituent of S a+f3 (X°) is an irreducible U(p, q) module. 

In other words, the extra degree of freedom in the U(p) x XJ(q) fibers has no effect 
whatsoever on the U(p, q) composition series of the S^iX ), except in so far as 
some fibers may be empty. We may therefore read off the V(p, q) constituents of 
the S a 'P(X°) from the corresponding results from 0(2p, 2g) (see Diagrams 2.25- 
28, 2.34, and 2.35), providing we take into account Diagram 4.16 describing the 
nontrivial fibers for S aJj {X°). 

Corollary 4.3. S^^A" ) is reducible if and only if a, (3 G Z, that is, if and only 
if a + (3 eZ and a + (3 = a — [3 mod 2. 
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Proof. For reducibility, we need one of the transition coefficients (2.20) to vanish 
for some (m, n) corresponding to a nonempty fiber of U(p) x U(q) modules. For 
example, the coefficient A~^~ 2q a+fj (m,n) [see definition (2.20)] will vanish if and 
only if 

a + (3 = m- n - 2(q - 1). 

Since we know a — [3 G Z, and to — n = a — [3 mod 2, we see that a + (3 G Z and 
a + (3 = a — [3 mod 2. This implies a, (3 G Z. Conversely, if a, /3 € Z, then we can 
reverse the process to find (to, n) such that A + ~ (to, n) = 0, implying reducibility. 
The same conclusion follows by considering A~ + , and A ++ and A~~ also cannot 
cause reducibility unless a, (3 G Z. □ 

To describe what happens when a and /3 are integers, we simply have to combine 
the relevant diagrams from §2 with Diagram 4.f6. The various possibilities are 
described in the following series of diagrams. 

(i) a, (3 G Z + . When a, /3 are both nonnegative integers, the barriers A^ , A 
and A ++ all intersect the if-type region and cause S a ^(X°) to break into four 
parts: one finite-dimensional subrepresentation, two quotients, and one constituent 
that is neither a quotient nor a subrepresentation (see Diagram 4.20). 



A-+ 




|a-/3| a + f3 + 2q-2 



Diagram 4.20. a e Z+, f3 g Z+. 

(ii) a or /3 is negative, but a + > 1 — p — q. Here the A ++ barrier no 
longer intersects the region of K- types, and there are at most three constituents, 
all infinite dimensional. As a + (3 decreases while a — (3 remains fixed, the barriers 
A^ and A ^ approach each other. When a and f3 are nonnegative, the point 
where the A^ barrier leaves the -ftT-type region, that is, the intersection of the 
A H barrier with the boundary of the .K'-type region, is on the x-axis. Similarly, 
the barrier A h passes through the boundary on the y-axis. As a + (3 decreases, 
A^ moves left and A ^ moves right. When a + (3 becomes sufficiently small 
(precisely, when min(a,/3) < 1 — q), the barrier A^ leaves the K-type region 
through the antidiagonal segment {x + y = \a — (3\ \ x > 0, y > 0}. As a + (3 
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continues to decrease, the point of exit of moves up the antidiagonal segment 
and finally (when max(a, (3) < 1 — (?) moves onto the y-axis. The barrier A h moves 
in the opposite direction. Its point of exit crosses from the y-axis to the antidiagonal 
segment when min(a, (3) = 1 — p and further crosses from the antidiagonal to the 
x-axis when max(a, (3) = 1 — p. The two barriers meet when a + /3 = 2 — p — q; 
their common point of exit from the K-type region is on the antidiagonal segment 
if and only if \p — q\ < \a — (3\. The real point on the unitary axis (see §4.4) is at 
a + {3 = 1—p — q. For a given value of a — (3, exactly one of the values 2—p — q and 
1— p — q for a +(3 will be a point of reducibility. If the unitary point a+/3 = 1—p — q 
is reducible, then S a ^(X ) breaks into a sum of two irreducible representations, as 
illustrated in Diagram 2.26. An illustrative situation among the several possibilities 
detailed above is pictured in Diagram 4.21. 




|a-/3| 

Diagram 4.21. 1 - p < min(a, (3) <1 — q. 

(hi) a + (3 < 1 — p — q. In analogy with the remark following Lemma 3.1, the 
modules S aJ3 (X°) and S a ' J) '(X°), where a + a 1 = 1 -p - q = (3 + (3', are naturally 
dual, so the structure of S a '^(X°) when a + f3 < 1 — p — q is dcduciblc from that 
of S a (X°) as described above. 

4.4. Unitarity. The unitarity criteria (3.6) for 0(2p, 2q) apply directly here and 
the results are as summarized in Diagrams 3.7 and 3.13. The unitary axis is defined 
by Re(a+(3) = 1—p—q. The possibilities for the values of a+(3 where the full module 
S a >P(X°) is unitary are as illustrated in Diagram 3.8 (note that a = a + [3+p+q—l, 
eo is + if p — q is odd, and S\ is — if p — q is even) . At a typical point of reducibility, 
as in Diagrams 4.20 and 4.21, the one or two constituents in the central strip are 
not unitary, while the constituents outside the barriers are unitary. Exceptions in 
the range Re(a + /3) > 1—p — q occur when a = (3 = 0, when the finite-dimensional 
subrepresentation is the trivial representation, and when a + (3 = 2 — p — q, with 
a, (3 € Z, in which case all three components, one of which has -ftT-types only along 
the line y — x = p — q, are unitary. See Diagram 3.9(a). 

4.5. The case of U(p, 1), p > 1. For U(p, 1), degeneration of the if-types leads 
to a situation substantially more complex than what is pictured in Diagram 4.16. 
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Since the module structure and unitarity result from an interaction between the 
if-types and the barriers, this change in K-type structure causes a corresponding 
change in the module structure and unitarity. 

As noted in the discussion following formula (4.9), the only H a ' b (C) that are 
nonzero are H n,0 (C) and H°' n (C), n G Z+. Hence, instead of what is repre- 
sented in Diagram 4.14 as a rectangle of K- types, we have only the if-types 
corresponding to the right and left sides of this rectangle, and in a given space 
S<*,P{X°) n j a+l3 (H m (C p ) <g> H n (C)), we will have at most two irreducible compo- 
nents for U(p) x U(l). 

This leads us to the following construction. Let us write 



(4.22) H n (C) = 



H n '°(C), n > 0, 

H°'- n (C), n < 0. 



Further let us represent j a +0 (H mi > m2 (C p ) ® H n (C)) by the point (mi + m 2 ,n) in 
R 2 . Here mi,m 2 G Z + and n G Z. Since 



j a+f 3 (n mi ' m2 (c p ) ® n n (C)) c s a 'p(x°), 



where 



(4.23) a - = m 1 - m 2 +■ 



and the triple (mi,m 2 ,n) is determined by the pair (mi + m 2 ,n) together with 
a — 0, we see that each point (mi + m 2 , n) represents at most one JsT-type in 
S a >P(X°). Furthermore, if the point (m,n), m G Z + , n G Z, does represent a 
if- type of 0(2p, 2q), then we must have 



mi = |(m — (n — (a — /?))) > 0, 
(4.24) m 2 = |(m+(n- (a -/?))) > 0, or 

m > \n — (a — 0)\. 



Conversely, as long as p > 1, this condition plus m + n = a — /? mod 2 guarantees 
we can find mi, m 2 , and n so that j Q+/3 (H mi ' m2 (C p ) ® W"(C)) C ^"^(X ) sits 
over (m, n). Thus the K-type region for S a - j3 (X°) is as depicted in Diagram 4.25. 




Note that the rc-intercept is no longer \a — (i\ but now is just a — (i. 



When we add barriers to this picture to determine reducibility, we find that each 
of the former barriers A ± ' ± produces two barriers, one in its original position and 
a second reflected through the a;- axis. Since when q = 1 the barriers A ++ and A + ~ 
intersect on the x-axis, the reflection of one continues the other in a straight line. 
The reflection of A h is A~~ and vice versa. Thus, with barriers inserted, Diagram 
4.25 is transformed into Diagram 4.26 (the shaded area is the if-type region). 
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Diagram 4.26 



Study of Diagram 4.26 and similar ones yields the following conclusions when a 
and (3 are integers. 



Lemma 4.4. Suppose a, [3 G Z. Consider the structure of S a '^(X°). 

(a) The barrier A ++ causes reducibility when j3 > 0. 

(b) The barrier A + ~ causes reducibility when a > 0. 

(c) The barrier A ^ causes reducibility when (3 < —p. 

(d) The barrier A~~ causes reducibility when a < —p. 



Given this information, the simplest way to organize the description of the struc- 
ture of S a ' l3 (X ) is by means of a picture describing the Hasse diagram of 5 Q,/3 (X°) 
in various regions of the (a, /3)-plane (see Diagram 4.27). The diagram is symmetric 
across the line a + [3 = — p because of the natural duality between S a ^(X°) and 

g-p-a-p-firx ). 
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Legend: • irreducibility and unitary 

□ finite-dimensional constituent 



Diagram 4.27. Hasse diagrams for S a 'P(X°), a, (3 in Z. 

Unitarity is determined again by the conditions (3.4). The main novel feature of 
the case at hand is that some of the conditions are now vacuous because there are 
no K-types corresponding to transitions that would prevent unitarity. Thus the 
unitary axis occurs at a + (3 = —p. If also \a — (3\ < p, that is, if a and (3 are in 
the central square of Diagram 4.27, then all barriers miss the K-type region, and 
if a + (3 varies while a — (3 is fixed, the barriers continue to miss the whole X -type 
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region in the interval — 2p + \a — (3\ < a + (3 < — \a — /3\. 

Consequently, the representation remains irreducible and unitary for a + [3 in 
this range. In other words, when \a — (3\ < p, the unitary point a + (3 = —p is 
irreducible regardless of the parity of \a — (3\ and the complementary series extends 
to a distance p — \a — (3\ on either side of the unitary axis: 



complementary 



series 



-2p+\a-/3\ 



\a-f3\ 



unitary axis: Re(a + (i) = — p 



Diagram 4.28. Complementary series for U(p, 1) 



In particular when a — (3 = [which is the case of the spherical principal series 
of U(p, 1)], the complementary series extends all the way from the unitary axis 
(a + (3 = —p) to a + [3 = 0, that is, a = (3 = 0, which contains the trivial 
representation as its irreducible subrepresentation. This has been known since 
Kostant [Ko]. 



complementary 



a = -p,(3=-p 



series 



trivial repn. 



-1 



a = 0,/3 = 



unitary axis: Re(a + (3) = — p 



Diagram 4.29. Complementary series for the spherical principal 
series of U(p, 1) 



At points of reducibility, some of the constituents may be unitary. When a, (3 € 
Z_|_, the "large" constituent that occupies the main part of the interior of the K- 
type region is unitary and the constituents living along the boundary of the i^-type 
region arc nonunitary (see Diagram 4.30). 
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Diagram 4.30 



Exceptions arise, however, when a constituent consists only of K-types on the 
boundary of the K-type region. In this case there are no transitions in the direction 
transverse to the boundary, so the corresponding obstructions to unitarity do not 
apply and the corresponding module is unitary. In Diagram 4.31 we describe this 
phenomenon for a, (3 > 0. 




a 



Diagram 4.31. The fillcd-in circles indicate the unitary constituents. 
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Similar remarks apply to the other regions of (a, (3) depicted in Diagram 4.27. 

Concluding remarks, (a) The unitary representations with AT-types along a bound- 
ary line of the if- type region have received considerable attention in the literature 
and are frequently called "ladder representations" [Ab, AFR, DGN, En, FR, SW]. 

(b) Consider the region of the (a, /3)-plane that is shaded in Diagram 4.32. 

For a given value of a — (3 G Z, the complementary series consists of precisely 
those values of (a, (3) that fall inside the shaded region. It is well known that the 
fundamental group of SU(p, 1) is Z. By considering representations of its universal 
covering group, one can get a family of representations parametrized by (a, (3) in 
which a — (3 can be an arbitrary real number. It is perhaps plausible that the set 
of (a, (3) for which these representations are unitary is exactly the shaded region 
(excluding its boundary). A. Koranyi (oral communication) has verified this for 
SU(2,1). 



a = — p 



a = 



("P-2,1) 



(0,0) /? = 



(1.-P) 
— • 



(3=-p 



{-P, ~P) 



(0,-p-l) 



(2,-p-l) 



Diagram 4.32 



(c) When \a— j3\ > p, the if- type region contains a parallel pair of barriers (either 
, A ^ or A ++ , A ) when a + (3 = —p is the real point on the unitary axis 
and the complementary series is restricted to the same region as when p, q > 1 (see 
Diagram 3.8). In this situation, however, we have the possibility that (3 < —p, a>0 
(or the reverse) so that at a point of rcducibility, both the barriers A^ and A ^ (or 
both of A ++ and A~~) intersect the if -type region and create submodules. These 
are the upper left and lower right regions in Diagram 4.27. When this happens, 
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the constituents outside the barriers are unitary, and one of them (for example, the 
region above the barrier A h when (3 < — p < a + (3) will have K- types contained in 
a diagonal strip of finite width (above A~ + and below the boundary of the K-type 
region). Thus we obtain for U(p, 1) a family of "laddcrlike" representations whose 
K- types live not on a single diagonal line but in a diagonal strip. Also we should 
note that when the real point on the unitary axis is reducible (i.e., when a — (3 = 
mod 2), the two constituents are very different: one is laddcrlike, with if-types in 
a finite strip, and the other has if -types ranging over a full quarter plane. 

5. The Sp(p, <?)-MODULE structure of homogeneous functions on light 

CONES 

5.1. The Sp(p, q) modules ^(X ). Let H denote the algebra of quaternions, 
and let M p+q ~ W © M q denote the vector space over H consisting of tuples 

LO = (u,v) = (ui, ... ,U p ,Vi,... ,V q ), Ul,V t e H. 

We consider M p+q to be endowed with the indefinite Hermitian form 

v <? 

(5.1) (u,w') Piq = {U,u') p - {V,v') q = ^U[UI - ^V[VI. 

1=1 1=1 

Here x denotes quaternionic conjugate of x E H. Also, since HI is noncommutative 
we should specify that we consider H p+<? as a right vector space over H. Specifically, 
quaternionic scalar multiplication multiplies the coordinate ofwelon the right: 

uih = (uih, u 2 h, . . . , u p h, Vih, . . . , v q h), h EM. 

This is the reason we write u[ui rather than uiu[ in formula (5.1). With this 
convention, the algebra End(H p+9 ) may be realized as the space of (p + q) x (p + 
q) matrices with entries in H, acting on u) E M p+q by the usual rules of matrix 
multiplication. 
Let 



(5.2) X° = {lo E W +q - 

be the light cone in M p+q . If i, j and k = 
we can write 



{0}|(w,w) M = 0} 

ij denote the standard quaternion units, 



Ul = Z 4 ;_ 3 + ix Al _ 2 +j(xu-l + ix 4i ) = Z 2 l-i + j Z 2l 

and 

vi = U41-3 + iyu-2 +j(yn-i + iyu) = w 2 i-i +jw 2 i- 

We may use the Xk and yk to identify H p+9 with R 4p+4q or the Zk and Wk to identify 
W +q with C 2p+2q . Under these identifications the light cone X° is exactly the same 
light cone as previously defined for M. 4p+4q or for C 2p+2q . 

Let Sp(p, q) C GL(p + q, H) be the isometry group of the Hermitian form (•, -^g. 
The identifications of the previous paragraph give us embeddings 

Sp(p,q)cV(2p,2q)cO(4pAq). 
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In particular, the group Sp(p,q) preserves the light cone X°. In fact, it acts tran- 
sitively on X° [Ja] . 

We can deduce in the usual fashion an action of Sp(p, q) on functions on X°, 
that is, 

(5.3) (g ■ - /(.g- 1 ^), g G Sp(p,g), w e ff>+«. 

The quaternionic scalar dilations H x also acts on X°, and on functions on X°, and 
this action of H x commutes with the action of Sp(p, q), so we can use H x to define 
invariant subspaces for Sp(p, q). 

Since H x is not commutative, we must consider H x -isotypic spaces instead of 
looking at only eigenspaces. Specifically, write H x ~ M x • Hi where Hi ~ Sp(l) ~ 
SU(2) is the group of quaternions of norm 1. For each integer j > 0, there is a 
unique (up to equivalence) irreducible representation Oj of Hi of dimension j + 1. 
Realize <jj on a space Yj. For a g C, define a representation er" of H x on Yj by 

a?(h) = \h\ a (Tj(h/\h\), heU x , 

where \h\ denotes the quaternionic norm of h. The representation Uj of Hi is self- 
dual. Consequently, there is a nondegenerate bilinear form (•, •) . (symmetric if j is 
even, skew-symmetric if j is odd) satisfying 

(cr«(h)y,y')j - (y,a](h)y')j, y,y' g Yj, h e H. 

Define a space Sj(X°; Yj) of smooth Yj- valued functions on X° by the recipe 

S^(X°;Yj) = {/ g C°°(X°;Yj) \ f(cuh) = a? (h) f (lo) , h g H x , u> G X }. 
Define also a map 

/? : SJpf ; Yj) <g> — -» C 00 ^ ) 

by 

We may define actions of Sp(p, g) on S?pT°; Yj) C C 00 ^ ; Yj) and on C°°(X°) 
by means of formula (5.3). It is trivial to check that (3 intertwines these actions. 
Furthermore, we may compute 

0(f®y)(uh) = (f(uh),y) j 

= {f{w),a a j {h)y) j 

= (3(f®^(h)(y))(u;). 

Let us denote the image of (5 by Sj(X°). It is not difficult to check that j3 is a linear 
isomorphism from S?(X°;Yj) <g) Yj to S<*{X°). Formula (5.4) therefore shows that 
the joint action of Sp(p, g) x H x on Sj(X°) is isomorphic to the tensor product of 
the action of formula (5.3) of Sp(p, g) on Sj(X°;Yj) with the action of cr" of H x 
on Yj. 
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Formula (5.4) implies that Sf(X°) C S a (X°). Here S a (X°) is as in formula 
(2.7). Standard arguments from harmonic analysis on compact groups tell us that 

s a (x°) = J2sj(x°), 

j>0 

the sum being in the sense of topological vector spaces. We should also note that 

S a+ (X a ) = S«(X°) 

j>0 
j even 

and 

s a -(x°) = s-(x°). 

j>0 
j odd 

We call Sj(X°) the er" -isotypic component of C°°(X ). We want to describe the 
Sj(X°) as modules for Sp(p, q). According to our construction of Sj(X°), this is 
essentially equivalent to describing Sj(X°;Yj) as an Sp(p, q) module. 

Our procedure for analyzing the Sp(p, g)-module structure of Sj(X°;Yj) will 
follow the pattern used before in dealing with 0(p,q) and \J(p, q). Indeed, since 
Sp(p, q) may be regarded as a subgroup of U(2p, 2q) and 0(4p, 4g), much of what 
we need is already done. We only give here the additional arguments that are 
required. In §5.2 we analyze the if-type structure, in §5.3 we discuss composition 
series and unitarity in the case p, q > 1, and in §5.4 we discuss the groups Sp(p, 1), 
which are particularly interesting because of degeneracy in the K-types similar to 
that already described for U(p, 1) in §4.5. 

Before beginning our detailed description of Sj(X°;Yj), we note that they can 
be thought of as induced representations, as for the representations of U(p, q) and 
0(p, q) previously studied. Precisely, fix a point loq €E X° and let Q be the stabilizer 
in Sp(p, q) of Ldo- Let P be the stabilizer of the quaternionic line ujqM through w - 
Define a map a : P — > H x by the formula puiQ = ui a(p). The map a is, in fact, 
a group isomorphism a : P/Q — > H x . Given any representation cr" of H x , the 
composition o a defines a representation of P. It is straightforward to check 
that the space Sj(X a ; Yj) is isomorphic as an Sp(p, q) module to the representation 
induced from aj a oaofF. 

Remark. When p = q = 1, the above representations constitute all principal series 
representations, so our analysis gives a complete account of the principal series and 
of the unitary dual of Sp(l, 1). Since Sp(l, 1) ~ Spin(4, 1) is the two-fold cover of 
the identity component of 0(4, 1), this case reproduces the calculations of Dixmier 
[Di]. 

5.2. ^-structure of Sf(X°). To investigate the structure of S°*(X°) as Sp(p,q) 
module, we first consider its if-structure. Here K will designate the maximal 
compact subgroup Sp(p) x Sp(q) of Sp(p, q) consisting of elements of Sp(p, q) that 
stabilize the decomposition W +q ~ H^QlHI 9 . Our experience in §§2-4 suggests that 
we try to extract an Sp(p) x Hi decomposition of H m (R 4p ). We start with some 
notation. 
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Let H(W) = {/ e V{W) I A/ = 0}. Here A is the Laplacian with respect to 
the real coordinate system defined after formula (5.2). We observe that H(M P ) is 
precisely the U(2p) harmonics in "P(C 2p ) and is also the 0(4p) harmonics in 7 7 (R 4p ). 

It is well known (see [Zh]) that irreducible finite-dimensional representations of 
Sp(p) can be indexed (using the highest weight) by ap-tuple of integers (£i, . . . , £ p ) 
with £i > £ 2 > • • • > C P > 0. We shall write a representation of highest weight 
0, . . . ,0) as Vp^ 1 '^ [note that the subscript indicates the group Sp(p)]. For 
Hi ~ Sp(l) ~ SU(2), we denote by Vf, the unique irreducible representation of 
SU(2) of dimension k + 1. 

Proposition 5.1. FFe feaue the following decomposition for H m (R 4p ) as Sp(p) x 
Hi ~ Sp(p) x Sp(l) module: 



(5.5) H n 



5l+?2=m 

Sp(p)xH 1 ym^ym p = 1. 



Hence, as Sp(p) x Hi module, 

(5 6) C°°(§ 4p ~ 1 ') ~ /^Ci>«2>o ^> £l ' £2 ® ^i 6 ?2 : P > 1) 

\E ro >o^r®^r. p = 1 - 

Proof. See §6.3. □ 

Assume for the moment that p,q>2. Combining the decomposition (5.5) for p 
and for q and an application of Clcbsch Gordan Formula [Zh] yields 



u m (w p )®n r 



(5.7) 6>«2>o, Ci+?2=™ 



3, Cl+?2=m 



0<j<m+n 

where the sum of the terms in braces is over 

£i > & >0, m>V2> 0, 
6 + 6 =m, r/i + 7] 2 = n, 

6 - 6 + m - m > j > 1(6 - 6) - fai - %)|, 
j = £i-& + m-m mod 2. 

Note that for a fixed j, there is exactly one copy of Vp^ 1 '^ V^ 7 ' 1 ' 7 ' 2 ^ ® V/ for each 
4-tuple £2, Vi> V2) satisfying (5.8). It is apparent now what the if-structure of 
S^{X°) is. 

Define, for each 4-tuple (£1, £2, 771, 772) satisfying (5.8), 



(5.9) 



52 



R. E. HOWE AND E. C. TAN 



where 

<t> e Vp (< ^ 2) ® V q ^™) ® v( c H m (R 4p ) <g> W n (K 4 «), 

2p 

r p = X! and a = ii+i2 + m + m + 2a - 
z=i 

The following statement is clear from the discussion above. 

Lemma 5.2. As an Sp(p) x Sp(g) x Hi module, Sj(X°) decomposes into a direct 
sum 

s«(x°) = ]T ^aAv^ 2) ® ® v/), 

w/iere ffte sum is over the set described in (5.8) (see Diagram 5.10). 

For reasons that will be apparent after the description of the transition properties 
of p C sp(p, q), we continue to organize if -types of Sj(X°) by means of the points 
(m,n) = (£i +£2,f?i +V2) in (Z + ) 2 . In other words, the if -types attached to (m,n) 
are those satisfying (5.8). We represent the 4-tuples (£1, £2, t?i, %) satisfying (5.8) 
as the integer points (£1 — £2, Vi ~ V2) in an auxiliary rectangle 

Rrn.n = { (x, y) | < X < TO, < y < 71 }. 

These points fill a coset of (2Z) 2 in a strip along the diagonal line x — y, as pictured 
in Diagram 5.10. We refer to these if -types as the fiber over (m, n). 



Vi -m (n- j, n) 




j m 



Diagram 5.10. Fiber of if -types over (to, n) (if j = m + 1 mod 2). 

Diagram 5.10 helps us to see that the fiber over (to,7i) is nonempty if and only if 

(i) to + n = j mod 2, 

(5.11) 

(n) to + n > j. 

Thus, when p > q > 2, the if -types of S°-(X°) fall in the region depicted in Diagram 
5.12. 



DEGENERATE PRINCIPAL SERIES 



53 




Diagram 5.12. if -type region of S°"(X°) whenp,q > 2 [consisting 
of points (m, n) where m + n = j mod 2]. 



Now consider the case p > q = 1. The computation parallel to (5.7) is 



£i>£2>0. 

£l+?2=m 



E (EW* 1 ^®*? 1 })®^ 



0<j<m+n 



where the sum of the terms in braces is over 



n > 0, £i > & > 0, 6 + 6 = m, 
6-6+n>i>|(6-6)-n|, 
6 - 6 + n = j mod 2. 



We again associate this set of if -types of Sj(X°) to (m,n) G (Z+) 2 - If we label 
these if -types in the fiber over (m, n) by £i — £ 2 = h then we obtain all I such that 



(5.13) 



| j — n\ < I < min(m,j + n) 



and congruent to j + n modulo 2. This is simply the set of if- types on the line 
rji — r]2 = n in Diagram 5.10. So if p > q = 1, the set of points (m, n) such that the 
associated fiber of if -types is nonempty can be depicted by Diagram 5.14. 
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y - X = J 




DIAGRAM 5.14. K-type region for S°-(X°) when p > q = 1 [con- 
sisting of points (to, n) where to + n = j mod 2]. 



For the case p = q = 1, the same arguments show that the if -types of Sj(X°) 



are V™ ® V™, where 
(5.15) 

The if -type region is then as depicted in Diagram 5.16. 



m + n > j > \m — n\, 
m + n = j mod 2. 




Diagram 5.16. if -type region for Sj(X°) when p = q = 1 [con- 
sisting of points (to, n) where m + n = j mod 2]. 



Furthermore, in this case the fiber over (to, n) consists of a single if -type, viz., 
Vi m <g> V". 
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5.3. Composition structure and unitarity. Since the technique involved in 
the consideration of unitarity is the same as in §§3 and 4.4, we will simply state 
the results alongside our discussion of the composition series structure. Write 



a,j 



a ,j (V^ 1 ^) V (m,m) y/) if p,q > 1, 
V^ 2 ' V) =i a ,j (v&M ® V? ® Vf) if p > g = 1, 
=i«,i ( vf ® ® I? ) if p = ? = 1 , 



and let 



2^i+&=m V a>J - , P ^ <7 > 1, 

>Jl+>?2=n 

V T/(£i,€2,n) „ >a -l 

be the sum of if-types of Sj(X°) living in the fiber over (to, n). We shall discuss 
the cases p > q > 2, leaving the interesting cases p > q = 1 to §5.4. Recall the 
functions A^ a from §2 [see (2.20)]. In parallel with the cases of o(p, q) and u(p, g), 
we let p be the space complementary to (sp(p) ©sp(g))c in sp(p, q)c- The following 
lemma is clear from the decomposition in (5.7) and Lemma 2.3. 

Lemma 5.3. There exist maps, nonzero whenever the target is nonzero, 

T ±± . M fa jj(m,n) TJ (m±l,n±l) 
1 m,n ■ P 29 U a,j ~* U a,j 

such that the action of z 6 p on a K-type is given by 

p{z)<j) =A++ 4 , >0 (& + 6, r?i + m+ „ 2 (z ® 0) 

+ ^tT4 9 ,a(6 + 6, m + %)T+; ?2i ^^(2 ® 0) 

+ ^4pT 49 ,a(Cl +&,m+ m)T^X^ m+m {z ® 0) 

+ ^m.atei +6, m + ^)7^- €2i , 1+J!!1 (« ® 0), 

w/iere £ ^(J^,m,>72) c 

So the action of Sp(p, q) [c u(2p, 2g) C o(4p, Aq)] on each fiber over (to, n) is 
compatible with Diagram 2.19. Since each fiber has several if-types, however, in 
order to have a grasp on the sp-module structure we must calculate how the sp(p, q) 
action affects the various points in each fiber. This is the essence of the next lemma, 
which we will prove in §6.3. 

Lemma 5.4. Consider a fiber U^™' 71 ^ of K -types in Sj (X°) . By successive applica- 
tions of the maps T m 7^ n+1 , T++, we can move from any K-type in ujj™' n ^ to any 
other. Hence, if the transition coefficients A^ p + 4qa (m,n) and A 4p4qa (to+1, n+1) 
are both nonzero, application of appropriate elements of the enveloping algebra il(sp) 
allows us to move from any K-type in uj^' n ^ to any other. Similar remarks apply 
to transitions from U^' n ^ to any of the adjacent fibers £/ ( (™ ±1 >™ ±1 ) i 
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Remark. The adjoint action of K on p makes p an irreducible if -module. The K- 
types that can be reached from v^&vum) by 

one application of p are isomorphic 
to constituents of V~ = p ® yfa&ww) _ g mce ^ nc X-weights of p are of the form 
±(ej ± /j) (see §6.3 for notation), it is not difficult to see the if-types that are 
embeddable in V~ are among the 16 whose highest weights are obtained by adding 
one of the vectors (±1,0, ±1,0), (±1,0,0,±1), (0,±1,±1,0), or (0,±1,0,±1) to 
(£i, £2, ^l, V2)- We will see that by moving from ujj™' n>> to {/(™+ 1 > n+1 ) by means of 

p and then back again we can move from v^'^ 2 ' Vl ' V2>> to any of its eight nearest 
neighbors, as illustrated in Diagram 5.17 (see also Diagram 5.10). 




Diagram 5.17 

In this diagram, the labels on the arrows indicate increments to (£i,£2, [0]r7i, 772) 
and the arrows show which increments can be achieved by applying T~+ x „ +1 T++ • 
Thus the arrow to (1,-1,0,0) indicates that we can move from y^'^ 2 ' 711 '^) ^ 
V^j +1 '^ 2 ~ 1 ' Vl,r,2>> and so forth. Similar pictures apply to the other transitions. Of 
course, if (£1, £2,771,772) lies on the boundary of the if -type region (see Diagram 
5.10), then transitions that would move it outside the region are suppressed. 

Study of Diagrams 5.10 and 5.17 will convince the reader that there are no 
barriers to movement between if-types in a given fiber, as long as one can move 
back and forth between the fiber and an adjacent fiber, but from the geometry of 
the barriers (see Diagram 2.22) we can see that the only occasion on which one 
cannot pass between a fiber and an adjacent one is for Uq ^ — and this consists 
only of the trivial if-type, so there is nothing to prove in this case. Thus by using 
Lemma 5.4, we can derive the following result. 

Theorem 5.5. (a) The image in any 0(Ap,4q)-irreducible component of S a (X°) 
of a given Sj(X°) is an irreducible module for Sp(p, q) x Hi. In particular, if 
p > q > 1, then Sj(X°) is reducible if and only if a E Z and a = j mod 2. 
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(b) Sj(X°) will have Hermitian constituents if and only if a is on the unitary 
axis (in which case Sj(X°) will be unitary) or a is real (in which case all its 
constituents will be Hermitian). A given Sp(p,q) constituent of Sf(X°) is unitary 
if and only if the 0(4p, Aq) transitions between the nonempty fibers of the Sp(p, q) 
constituent are consistent with unitarity. 



Remark. As in the case of U(p, q) modules S a - j3 (X a ), one obtains an Sp(p, q) 
composition series for an Sp(l) eigenspace of S a (X°) by intersecting it with the 
0(4p, Aq) composition series for S a (X°). 



Proof. Again this follows essentially from Lemmas 5.3 and 5.4. The pictures when 
P > q > 2 will be similar to those under Case EE in §§2 and 3 (see Diagrams 3.13, 
4.20, and 4.21). □ 



5.4. The case of Sp(p, 1). The if-typc regions are depicted in Diagrams 5.14 and 
5.16. For p > q — 1, there is a "line" of K-types over each point in the -ftT-type 
region. We note that unlike the situation of U(p, 1), if-types along each fiber can 
be moved to adjacent ones via operators in il(sp(p, 1)). The analogs of Lemmas 5.3 
and 5.4 hold. We do not state them formally. The following result is easy. 



Theorem 5.6. Assume p > q = 1. // a £ 7L or a ^ j mod 2, then Sj(X°) is 
irreducible for Sp(p, 1). 



Following §4.5, we organize our description of the structure of Sp(p, 1), p > 1, 
by means of a picture (see Diagram 5.18) describing the Hasse diagrams of Sj(X°) 
in the (a, j)-planc (a G R). Observe that the diagram is symmetric about the 
axis a = —2p — 1, because of the duality between the modules S~ 2p ~ 1+s (X°) and 

S~ 2p ~ 1 ~ s (X°) (see Lemma 3.1). 
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□ 



nommitary constituent 
unitary constituent 
finite-dimensional constituent 
trivial representation 



Diagram 5.18. Submodule structure of Sj(X°) when a 
mod 2 for Sp(p, 1). 



Again the existence of the complementary series is very interesting. Notice that 
in the triangular region given by a — j > — 2 — 4p and a + j < —2, the modules 
are irreducible and unitarizable. The length of a horizontal line in this triangle 
will be twice the length of the complementary series for a fixed j, that is, there 
is a complementary series of length 2p — j — 1 for each j < 2p — 2. The extreme 
case when j = 0, that is, the spherical principal series, has complementary series 
of length 2p — 1. Unlike the case for U(2p, 1) module S a '^(X°), where a = (3, the 
complementary series does not run till a = 0. This can be explained by observing 
that, for U(p, 1), the barriers A ++ and A A always intersect on the x-axis, while 
for Sp(p, 1) there is a gap of length 2 between the x-intercepts of the A ++ and 
barriers. This means that as o increases from the unitary axis it is always 
the A H barrier that first causes reducibility and prevents further unitarity, even 
in the extreme case j = 0. 

Diagram 5.19 describes the situation for Sp(l,l), with the same legends as in 
Diagram 5.18. We hope it will be self-explanatory. 
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unitary axis 

a = -3 




Legend: o nonunitary constituent 

# unitary constituent 

■ ladder representation 

□ finite-dimensional constituent 

□ trivial representation 



DIAGRAM 5.19. Submodule structure of S<*(X°) for Sp(l, 1). 



Remarks. We add some remarks in the case of Sp(2, 1). Our results showed that 
the if-types of S?(X°) are of the form V^ ui2) <8> V? where fr, n satisfies (5.13) 
(see Diagram 5.12). Let us instead plot the second and third coordinates, that 
is, (^2,n), fixing £i; this is a section of the if -type region parallel to the £i axis 
(see Diagram 5.20; here £i (> j) is fixed and we also have the parity condition 
£i +f 2 +ne 2Z). 

The following statements are based on the second author's thesis (see [Ta]), and 
we refer our readers to that for full details. We will be somewhat sketchy. If 
we consider arbitrary principal series representations of Sp(2,l), we get highest 
weights (£,i,£,2,n) that live in regions whose convex hulls are wedges in R 3 . If we 
take sections parallel to the £i-axis, we get Diagram 5.21 (note that Diagram 5.20 
is a "degenerate" case of this diagram). 
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n 



(£i-j,2j) 




Diagram 5.21 



The multiplicities of iiT-typcs along the boundary of the figure is one, and the 
multiplicities "grow" linearly with the distance from the boundary, attaining a con- 
stant towards the 'central' region, much like the way the weight multiplicities of an 
irreducible s[(3) representation behave in relation to its weight diagram. Further, if 
we look at K-type diagrams for the quotients of the principal series representations, 
we get diagrams like Diagram 5.22. 
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Diagram 5.22 



The multiplicities behave as described in the previous paragraph. Details are omit- 
ted since the results arise from painstaking computations involving Blattner's For- 
mula, Branching Rules, and known composition series structure for Sp(2,l). A 
reason for the nice multiplicity result in this case may be the already apparent 
linearity in the branching rules for Sp(2) x Sp(l) [a maximal compact subgroup of 
Sp(2,l)] toSp(l) xSp(l). 



6. Technicalities 

6.1. Computations for 0(p, q). Let x\,.. . ,x n be the standard system of coor- 
dinates in W 1 and let 

j=l 3 2 = 1 j = l ■> 

We have the commutator 

[A, r 2 ] = AE + 2n 

and so {A, r 2 , AE + 2n} span a Lie algebra isomorphic to sl(2, R). If P is a homo- 
geneous polynomial in V(M. n ), let degP = E(P) be the homogeneous degree of P. 
Then 



(6.1) 



A(r 2 P) = r 2 AP + {AE + 2n)P 

= r 2 AP+(AdcgP + 2n)P. 



There are unique harmonic polynomials hi such that 
(6.2) P = J2hi(r 2 r 
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for some positive integer I and degP — deg hi + 2dj. To compute the /ij's, we use 
(6.1) inductively. If AP = 0, then I = 1, hi = P, and on = 0. If A 2 P = 0, compute 

A(r 2 AP) =r 2 A 2 P + (4(dcgP - 2) + 2n)AP 

^A|P-- Uo 

4(degP-2) + 2ny 



and get 

(6.3) P = P~ + r 2 



AP 



4(degP-2) + 2ny ' 



where for convenience we will write 

r 2 AP 



P~ = P- 



4(dcgP-2) + 2n' 



(It should be clear from context which space 1" we are working with when the 
definition above is being used.) One could proceed inductively using (6.1) to obtain 
the decomposition in (6.2) for any homogeneous P. 
If (j) = h! <g> h 2 G H m (W) <8> W"(R9), define 

T++cj>={x i h l r{y j h 2 r, 
9/ii 9/i2 



where 



1 , 1 

ci = — — and c 2 



(2deg/ii-2+p) (2deg/i 2 - 2 + g)" 

Extend the above definitions linearly to all elements in K m (W) <8> W n (K«). 

Proof of Lemma 2.3. Suffices to show for 

d d 
z = Xi—^-yj— e p. 
dyj OXi 

Thus, for h = h!h 2 rp where hi G H m (R p ) and h 2 G compute 



From (6.3) we have 

9x 



arifti = + ci ( ) r 2 
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and 



yjh 2 = {yjh 2 )~ + c 2 I ^ ] r 2 = (yjh 2 )~ + c 2 ( ^ ^ " : 



dyjj q \dy 



since r 2 = r 2 on X°. Hence, 



r p i r p 



+ 2 7 jOr^r + ci J r 2 p | | (y^r + c 2 

which gives the required result if p > q > 2. □ 



Proof of Lemma 3.1. We will first show that if dx, dy are the rotation-invariant 
probability measures on S p_1 and S 9 " 1 respectively, then 



J(/) = / f(x,y)dxdy 



/Sp- 1 XS9- 1 

is an 0(p, g)-invariant linear functional on S 2 ~ p ~ q (X°). To this effect, let 

X? = {{x,y)eX°\i<r p (x)<s}, s > 0. 

The 0(p,g)-invariant measure on X° ~ S^" 1 x^" 1 xM£ is tP +q - 3 dx dy dt. Observe 
that for / e ^-p-^X ) 

210^ / x o >< te > *> dX dy dt = 2^S ( I *) J ^ = J ^ • 
If .g G 0(p, g), the image of X® under g is 

gX° = { (tV.tV) e X° I z' e S^ 1 , y' e S*" 1 , < f < s^y') }, 

where (f>(x',y') is a smooth function on x Thus, 

Ag- 1 ■ f) I f(t'x', t'y') t' {p+q - 3) dx' dy' dt' 

2 lOgS j g . X 

s<p(x',y') ../\ 



2 logs 7sp-i x s<!-i \J±{£_^i t 

= I f(x',y')dx'dy' 



= Af), 

which proves our earlier assertion. 

For / G S- a+2 -P-"{X°) and /i e S"(X°) define 



(/, ft) = J (fh) = / /(a;, y)h(x, y) dx dy. 

JSp- 1 xS?- 1 
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Then it is easy to see that this is an 0(p, ^-invariant Hermitian pairing between 
S- a + 2 -P-i(X°) with S"{X°). It gives a Hermitian inner product on S a (X°) if 

-a + 2-p-q = a or Rea = + + i. 
This proves Lemma 3.1. □ 

6.2. Computations for U(p,q). Define for Sij and Tij as in (4.18) and <j> = 



(6.4) 



where 



S 1U01 (5 y ® (hi ® ft 2 )) - (z 4 /n)~(T^/i 2 )~, 

^°- 101 (% ® (/ii ® ft 2 )) - c^iwjh*)-, 

r)h 

S 10 - 10 (S l3 ® (ft! ® fc 2 )) = ciiz^r—?-, 

-.0-1-10/ o ^ ^ ,. « _ 9/l 2 



'(Sy (gi (/ii ® ft 2 )) = cic 2 



r °(Tij ® (/ii ® /i 2 )) = (^/nrK-^r. 

T- 1010 (T y - ® (/n ® h 2 )) = d-^-iwjhzr, 
T mu - i (T ij ®(h 1 ®hi))=c 2 {zih 1 ) 



01,1 , %®(/ii®/ l2 )) = c 2 (^ 1 )~^, 
-ioo-i/^.. ^ ~ fc _ ^^dhidhv 



dzi dwj 



_ 1 

(p + mi + m 2 - 1) ' 
1 

(g + m + n 2 — 1) ' 



" + de Sn,2 P + de Sn,2 P - 1 

[Note that P~ can be shown to be harmonic if A^P = through computations 
similar to those before (6.3). Here deg n z P and deg„ f P are the degrees in z and z of 
PmV{C n ).] Extend the definitions (6.4) linearly to all of H mi ' m * (C p )®W ni <™ 2 (C). 
Lemma 4.1 then follows from a straightforward computation as in Lemma 2.3. 

We still need to verify that one could move freely within each fiber (see the 
discussion after Diagram 4.19). Write [see (4.11)] 

^f, ro2 ,„ 1 ,„ 2) =i«,/3(w mi ' ro2 (c p )®w" 1 '" 2 (C 9 )). 

Let us show that there is a nonzero map U + from il(u(p, q)) with 
(6 5) U + ■ V"' 13 — > V a 'P 

V * / " (mi,m2,ni,7i2) (mi + l,m2- l,ni — 1,712 + 1) ' 
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This is a movement in the south-westerly direction along the fiber depicted in 
Diagrai 
that is, 



Diagram 4.15. Let nf" 11 *" 12 '" 1 -™ 2 ) be the projection to the AT-type V, a ' 

1 ■' J L (mi,m2,ni,n2)' 



(6 6) n (mi ' m2 '" 1,rt2) • S aJ3 (X a )—> V, 01 ' 

V / V / (mi,m2,Tii,n2) 

This projection certainly arises from the action of if on ^"'^(X ). Let 

TO = mi+m 2 , n = ni+n 2 



and write 
(6.7) 



for G ft mi - m2 (C p ) ® H ni '" 2 (C«). Then the maps 

(a) n(p(S ij ){Il("H+ 1 < ™ 2 > »i. " 2+1 V(^)(j a>/3 (0))}) 

= A—(m + 1, n + l)A++(m, n)^ 1 - 1 ^™ 1 ^)), 

(b) n( P (s ij ){n(™+ 1 > ™ 2 < ^piSiMcM))}), 

= A-+(m + 1, n - l)A+-(m,n)^- lol (4°- lo (0)), 

(c) n(p(Si J -){n (mi - m2 " 1 - ni - 1 - n3) p(Sn)(j a A<t>))}) 

= A++(m - 1, n - l)A~(m, n)^ 001 ^ 1 " 10 ^)), 

(d) n( P (^){n(™- ™ 2 -^ "i. 

= A+"(m - 1, n+ l)A-+(m,n)^°- lo (^ lol (0)), 

where 

n = n^™ 1 ^ 1 ' m2_1 ' "i -1 ' "2+1) 

have the mapping property as in (6.5). It suffices to show that at least one of 
them has nonzero coefficients. A simple check using formulas (2.20) verifies that 
all four coefficients vanish only when m = n = = a~P, but when this happens, 
V,"' 13 ,, ,„ vanishes, that is, the fiber at (0,0) has only one if-type. 

(mi±l, 7712 + 1, ni=Fl,TJ2±l) ' ' v ' ' ■' J r 

Thus, we can always move in the south-westerly direction along the fiber. The 
proof for the north-easterly movement is similar. 

6.3. Computations for Sp(p, q). Choose z\, . . . , Z2 P ,Wi, . . . ,u>2 9 ,zT, . . -,Z2 P , Wi, 
. . . , wig as a system of coordinates in (H. p+q )c- The complexified Lie algebra of 
Sp(p, q) may be described as the set of matrices in M 2p ,2q{C) 



An 


A12 


A13 


A14 


A 2 i 




A 23 


A 2 A 


A\a 


—A\ 4 


A33 


A34 


^23 


-^13 


A i3 


"^33 



where A n , A 12 , A 21 e M P (C), A 33 , A 34 , A 43 € M,(C), A13, A i4 , A 23 , A 24 £ 
M Ptq (C) with A12 = A* 12 , A21 = A 21 , A 34 = A\ A and A 43 = A^ 3 . Note that sp{p) c 
and sp(g)c are embedded block diagonally in sp(p, q)c- 

It is not difficult to check that the following first-order differential operators span 
the action of sp(p)c on the space of polynomial functions on (W +9 )c [1 < i,j < p 
for (a) and 1 < i < j < p for (b) and (c)]: 
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, . Al _ d d d d 

(a J A {j — zi— Zj—zi + z i+p — : - z j+p 



dzj d~zi dzj+p dzi+p 

d d d d 



(6 - 8) (b) B ^ = Zi ^ p - z ^m + Zi dz-^- Zi+ ^ 

, n n \ _ d _ d d _ d 

[c) c ^- z - j+ ^r Zi &zw P +Zi+p dz-r z °^r P ' 

Similarly, the following describes the action of sp(<?)c [1 < i,j < 5 for (a) and 
1 < i < J ' < 1 f° r (b) and (c)]: 



(a) A ^ = Wi ^r Wi ^ + Wi+q ^=r q ~ Wi+q ^ 

d _d_ d _d_ 

dwj +q 3+q duli 3 dw i+q t+q dwj' 



i+q 



d d d _ . _ d 



(c) C iS = w j+q T—_-w t —= + w 1+q — - w_ 



13 3 ^ q dw t dwj^ q d Wj 3 dW[^' 

Finally, the space p complementary to sp(p)c®sp(q)c in Sp(p, q)c is spanned by 
the following (1 < i < p and 1 < j < q): 

, , p d . d d . 9 

\a) rij — z. 



dwj dz p+ i dzi dw q+ j 

d d d d 



(6.10) 



(b) Qij = Zi d^- Wj d^i +Wq+j m~ Zp+i ^ 

, . TT d d d _ d 

(c) Uij = z p+l — - — + ttfj T== - «, 



9wj dzi dzp + i dw q+ j 
d d d d 



(d) Vij = Zp+i d^ +Wj d^ +Wq+j d^ + Zi W 



Let 



9 n \ - 5 



.12 



= E(^-^^=), and ^=£^_-^_j 



It is not difficult to check that {A p , r 2 , £n, E\ 2 , E 2 \, E 22 ) commutes with Sp(p) on 
the space of polynomial functions 

P(Hg) ~ C[zi, . . . , z 2p ,zj, . . . ,Zfy]. 
They span a Lie algebra isomorphic to 

(4)£ = e gZ( 4 ) ^ J + = and X J + JX = o} c , 
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where 

h " 
-I 2 ' 

Here I2 is the 2x2 identity matrix. Observe that {-En, E12, -£21, -E22} spans the Lie 
algebra gl(2) where Eij corresponds to the standard matrix unit at (i,j) in g[(2). 
In particular, one sees that the Lie algebra spanned by {En — E 2 2, E\ 2 , E 2 i} 
corresponds to the infinitesimal action of Hi ~ Sp(l). 

With the notation as in §5.2, we have the following result, which implies Propo- 
sition 5.1 immediately 



J 



h 
h 



and J 



Lemma 6.1. The decomposition (for p > 1) 



H{W) 



Sp(p)xHi 



runs through all representations Vp^ 1 '^ of Sp(p). The Sp(p) module Vp^ 1 '^ 
V^ 1- * 2 is generated by the joint highest weight vector 



(6.11) 



Z\ Zi +p 



Z2 Z2+p 

For p — 1 the isotypic decomposition of H(M) is 



W(H) 



Sp(l)xSp(l) 



J2 vr ® vr, 



m=0 



where V™ <g> V™ is generated by h\^ m = z™ . 



Remark. If we apply the lowering operator _E 2 i in sp(l) to the joint highest weight 
vector (6.11), we find that a spanning set of Sp(p) highest weight vectors in H(W) 
are given by 

U ) I^Ki-feW = z P+i 3h p,(ti-j,&)> j = 0, 1, £1 — £2, 

\hl, m ,j = Z2 J hl tm -j, J = 0, 1, 771. 

Proof. Choose the positive roots of sp(p)c to be those described by (6.8)(a) (for 
1 < i < j < p) and (6.8) (b). It is easy to check that h n j^^ are sp(p)c x Sp(l)c 
highest weight vectors of weight (£1, £2, 0,0,... , 0) <S> (£2 — £2)- For the remaining 
part of the first assertion, we refer to [Ho2]. □ 



Proof of Lemma 5.4. We proceed to compute the action of the p-part of Sp(p, q) on 
Sf{X a ). From Lemma 6.1, the Sp(p) x Sp(<?) highest weight vectors in n m {R ip ) <g> 
H"(R 4 «) are 

hp t (a 1 ,a 2 ),j^ l q.Xbi,b 2 ),k — 'hp, (di —3,0,1) w q+l hq, (61 -k,b 2 ) ■ 
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For fixed (01,02) and (61,62), these vectors span an sp(l) module isomorphic 



to Vf 



yo! 6 2 The Clebsch-Gordan formula tells us that the sp(l) highest 



weight vectors in this module are the functions 



^ l pAa.i-j-a 2 )^'q,(b 1 -j,b2) 



Zl 
Wl 



z p+l 



Wq+1 



To adjust the homogeneity of these functions, we should multiply by an appropriate 
power of ri. Thus we wish to study the functions 



7(01, a 2 , h, b 2 , c, d) = z\ 



Zl 


Z P+1 


W* 1 




Wq+1 


62 


Zl 


Zq+1 


Z 2 


z p+2 




w 2 


Wq+2 




Wl 


Wq+1 



Here aj, bj, and c are positive integers and d is a complex number. Note that our 
use here of aj, bj is slightly different from that of a few lines above. A check shows 
that 7(01, a 2 , bi, 62, c, d) is an Sp(p) x Sp(g) highest weight vector of Sp(p) weight 
(ai + a 2 + c, a2, 0, 0) and Sp(q) weight (61 + b 2 + c, 6 2 ,0,...,0). Also it is an 
sp(l) highest weight vector of weight ai + bi. 

Recall the notation in §5.3. The possible components in p&iVp^ 1 '^ <X> V^ 1 ''' 2 ^ are 
the sixteen components obtained by adding (±1, Q ±1, 0), (±1, Q 0, ±1), (0, ±1, ±1, 0) 
and (0, ±1, 0, ±1) to (£1, £2, %)• Our intention will be to show that we can move 
from the fiber over (m, n) to a neighboring fiber (mil, nil) and then back, and 
doing this a sufficient number of times to get to any other points in the original 
fiber. 

Because the moves between, say (£1, £2, ^2) to (£1 + 1,£2, r\i + 1, "2), and 
in the reverse direction arc essentially adjoint to one another (exactly so on the 
unitary axis), we can do one move if and only if we can do the reverse move. Thus 
it suffices to check if we can do eight of the sixteen moves. 

To show that we can do the (1,0,1,0) move, we apply Qn [see (6.10)] to one of 
our functions 7 : 

(6.13) Qn7(ai, a 2 , h, b 2 , c, d) = dj(ai, a 2 ,bi,b 2 , c+1, d-1). 

To show that we can do the (1,0,0,1) move, consider 



(6.14) 



(A^Qu - (bi + c + l)Qi2)7(oi> «2, h,b 2 , c, d) 



= -£>id7(ai + 1, a 2 , 61 - 1, b 2 + 1, c, d - 1). 
Similarly for the (0,1,1,0) move, we appeal to the formula 

^ 6 15 - (AliQn - (01 + c + l)Q 2 i)7(oi, «2, h,b 2 , c, d) 

= -aidj(ai — 1, a 2 + 1, 61 + 1, 6 2 , c, d - 1). 

Finally, for the (0,1,0,1) move, we can verify that the operator 

A\iA\ x Qn - (ai + c + l)^!<92i 

- (61 + c + l)A^gi2 + (ai + c + 1)(&! + c + i)g 22 

applied to 7(01, a 2 , bi, b 2 , c, d) yields 



(6.16) 



c(ai + bi + c+ l)d7(oi, a 2 + 1, 61, b 2 + 1, c-1, d-1). 
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Thus in passing from the fiber uj^' n ^ to the fiber jj( r ^+ 1 ' n+1 ) ( see §5.3 ; before 
Lemma 5.3) by means of the action of the Qij, we can add all four of the conceivable 
vectors (1, 0, 1, 0), (0, 1, 1, 0), (1, 0, 0, 1), and (0, 1, 0, 1) to a given highest weight of 
U^' n \ By taking adjoints, we can add the negatives of these vectors when passing 
from U^ +1 ' n+1) to Uj£' n) . Thus passing from U^' n) to U^ +1 ' n+1) an d back 

again allows us to move from a given if -type in U^' n ^ to any of its eight nearest 
neighbors, as illustrated in Diagram 5.17. 

If the vanishing of either of the transition coefficients A^ 4qa (m, n) or A±~ iqa (m+ 
1, n+1) prevents us from moving to £/(™+ 1 >™+ 1 ) or back to U^' n \ we can equally 
well move from u[ m , ,n ^ down to U^™ l ' n ^ and back up to achieve the same end. 

a >J a ;J 

Study of the diagrams of the if -type regions (Diagrams 5.12, 5.14, 5.16) and the 
positions of the barriers (Diagram 2.34) shows that the only time neither of these 
strategies works is when (to, n) is on the border of the if -type region; but in this 
case, we can see from Diagram 5.10 that the fiber contains only one if -type, 

so there is nothing to show. Although we do not need them, similar calculations may 
be done for the transitions from lli"]^ to jj^ +1 ' n ~ 1 '> anc j back. This concludes 
the proof of Lemma 5.4. □ 

Concluding remark. Although in this paper we have been exclusively concerned 
with questions of infinitesimal structure, our results have some implications for 
harmonic analysis on hyperbolic spaces (also describable as isotropic (reductive) 
symmetric spaces [Sc]). To describe these, consider the complex projective hyper- 
boloid Y = U(p, q)/(\J(p — l,q) x U(l)). Similar remarks apply to the quaternionic 
hyperboloid. As noted in [Sc], the real hyperboloid 

X! = {we R 2p+2q | (w,w) p , q = 1} 

fibers over Y with fiber U(l). In fact, this is a principle U(l)-bundle. It is well 
known (see the references on the first page, especially [Sc]) that all the representa- 
tions occurring in the harmonic analysis of the 0(2p, 2q) action on X\ are among 
those considered in §§2 and 3. Our results imply that the restriction of any one of 
these representations to U(p, q) decomposes discretely; more precisely, for any of 
these representations of 0(2p, 2q), the cigenspaces of the fiber U(l) of X\ over Y 
will be irreducible modules for U(p, q). It follows that the U(p, q) harmonic analysis 
of any line bundle constructed from the fibration X\ — ► Y reduces immediately 
to the 0(2p,2q) harmonic analysis on X\. In particular, harmonic analysis on Y 
reduces to harmonic analysis on X\. 

7. Afterword of caution 

In this article we have tried to exhibit some of the elegance of representation 
theory of semisimplc groups. We hope some readers will be inspired to go further, 
in the texts [Kn], [Wa], or the expository accounts [Ho4], [Vo4]. Here at the end, 
we should warn the reader otherwise unfamiliar with representation theory that 
the examples we have studied are still only a small, and in some ways misleadingly 
comprehensible, sample of the full spectrum of representations of semisimplc groups. 

One way to locate our representations in relation to the whole is to recall their 
description as degenerate principal series [see, e.g., formulas (2.9)-(2.11)]. They are 
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representations induced from certain finite-dimensional representations of the group 
Pi, the stabilizer of an isotropic line. This group is a maximal parabolic subgroup, 
and it is its maximality, and the corresponding smallness of the coset space G/Pi, 
that leads to the use of the adjective " degenerate" . If one considers representations 
induced from finite-dimensional irreducible representations of a minimal, rather 
than maximal, parabolic subgroup, then one has the (ordinary or nondegenerate) 
principal series. [For 0(p, q) a general parabolic subgroup is describable as the 
stabilizer of a nested sequence Xi C X2 C A3 C • • • C Xu of isotropic subspaces. If 
the sequence contains a subspace of every dimension from one to min(p,(7), then the 
parabolic stabilizing it is minimal. Similar remarks apply to U(p, q) and Sp(p, q).] 
Principal series have the following properties: 

(1) They come in continuous families, parametrized by several (in place of the 
single parameter of our examples) complex variables. 

(2) For generic values of the parameters, they are irreducible. 

(3) In general, they have finite composition series. 

(4) Any (reasonable) irreducible representation is equivalent (via an appropri- 
ate notion of equivalence) to a constituent of some principal series. 

Properties (l)-(3) are qualitatively similar to what we saw in this paper, and prop- 
erty (4) tells us that if we understand the principal series, then we understand 
an essential part, in particular the building blocks, of semisimple representation 
theory. 

It might seem that properties (l)-(4) virtually finish the subject, but no one 
has so far managed to achieve control of general principal series comparable to the 
picture given for the examples in this paper. The point is that the combinatorial 
structure of the composition series of a general principal series, and of its con- 
stituents, is rather spectacularly more complex than what we encountered in our 
examples. (See [CC] for a recent discussion.) Some of the most important advances 
in the subject (Langlands classification [Kn, Wa, Vo2]; Kazhdan-Lusztig conjec- 
tures [BB, BK, KL, Vo3]) have involved obtaining significant information about 
the composition series of principal series. 

Let us remark that when q = 1, our maximal parabolic is the minimal and 
only proper parabolic subgroup, and in this case our representations are ordinary, 
not degenerate, principal series. When p is also small, our representations in fact 
include all, or essentially all, principal series, so that we find all the irreducible 
representations, and since we know which are unitary, we classify the unitary dual. 
Specifically we obtain all principal series for U(l, 1) and for Sp(l, 1) (~ Spin(4, 1)), 
and we obtain all principal series up to twisting by a quasi-character of the whole 
group for 0(2, 1) and U(2, 1). 

Since, at least when p and q are greater than one, the representations we have 
considered are quite special, one may ask if any interest attaches to them, aside from 
the inherent attractiveness of the picture we have tried to present. In fact, these 
representations arise in studying harmonic analysis on hyperboloids. The full im- 
port of the term "harmonic analysis" has never been completely decided, but many 
would agree that it certainly includes the decomposition of L 2 -spaces of homoge- 
neous spaces. Among the homogeneous spaces of semisimple Lie groups, the class 
of "semisimple symmetric spaces" or "affine symmetric spaces" [F-J] recommends 
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itself as particularly interesting. The hyperboloids 

X t = {w e R p+q | (w, w) M = i] 

are very closely related to (precisely, are principal fibre bundles with compact fibre 
over) some of the simplest examples of semisimple symmetric spaces [Sc] , and the 
representations we have studied enable one to decompose the L 2 -spaces of hyper- 
boloids. The connection with harmonic analysis on semisimple symmetric spaces 
was the motivation for many of the papers cited in the introduction. 
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